Method for constructing scene template library, object prediction method and device, and electronic device

Abstract

The present disclosure provides a scene template library construction method, object prediction method and device, and electronic equipment, which can be applied to the fields of computer technology, big data technology and artificial intelligence technology. The scene template library construction method comprises: performing feature extraction on a sample object sequence to obtain a sample object feature vector; constructing an initial scene template corresponding to sample scene label information according to the sample scene label information corresponding to the sample object sequence; determining a sample object fusion vector according to the sample object feature vector and the initial scene template; adjusting the initial scene template according to the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information; and constructing a scene template library according to the sample scene label information and the target scene template corresponding to the sample scene label information.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer technology, big data technology, and artificial intelligence technology, and more specifically, to a method for constructing a scene template library, an object prediction method and device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of computer technology, how to process time series data for time series prediction is an urgent problem to be solved.

[0003] Time series data refers to data collected over time, which can be used to reflect the state or degree of change of an object or phenomenon over time. Time series forecasting refers to the analysis of future trends based on historical time series data.

[0004] In the process of realizing the concept of the present disclosure, the inventors discovered that there are at least the following problems in the related art: a single model cannot meet the needs of time series prediction based on time series data in different scenarios. Summary of the Invention

[0005] In view of this, the present disclosure provides a method for constructing a scene template library, an object prediction method and device, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of the present disclosure, a method for constructing a scene template library is provided, comprising:

[0007] Perform feature extraction on the sample object sequence to obtain the sample object feature vector;

[0008] Constructing an initial scene template corresponding to the sample scene label information according to the sample scene label information corresponding to the sample object sequence;

[0009] Determining a sample object fusion vector based on the sample object feature vector and the initial scene template;

[0010] Adjusting the initial scene template according to the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information; and

[0011] The scene template library is constructed based on the sample scene label information and the target scene template corresponding to the sample scene label information.

[0012] According to an embodiment of the present disclosure, the initial scene template includes a first initial scene matrix and a second initial scene matrix.

[0013] According to an embodiment of the present disclosure, determining the sample object fusion vector based on the sample object feature vector and the initial scene template includes:

[0014] Fusing the sample object feature vector and the first initial scene matrix to obtain a first output feature vector;

[0015] Encoding the first output feature vector to obtain an intermediate encoding vector;

[0016] Fusing the sample object feature vector and the second initial scene matrix to obtain a second output feature vector; and

[0017] The intermediate coding vector and the second output feature vector are decoded to obtain the sample object fusion vector.

[0018] According to an embodiment of the present disclosure, constructing the scene template library according to the sample scene label information and the target scene template corresponding to the sample scene label information includes:

[0019] Determine the above sample scene label information as scene template key information;

[0020] Determine the target scene template as scene template value information;

[0021] Constructing a scene template key-value relationship based on the scene template key information and the scene template value information; and

[0022] The above scene template key-value relationship is stored in the above scene template library.

[0023] According to an embodiment of the present disclosure, the above-mentioned feature extraction of the sample object sequence to obtain the sample object feature vector includes:

[0024] Inputting the sample object sequence into a feature extraction module in a pre-trained model to obtain the sample object feature vector, wherein the pre-trained model further includes a first fusion module, a first encoder, a second fusion module, and a first decoder;

[0025] The first output feature vector is obtained by processing the sample object feature vector and the first initial scene matrix using the first fusion module;

[0026] The intermediate encoding vector is obtained by processing the first output feature vector using the first encoder;

[0027] The second output feature vector is obtained by processing the sample object feature vector and the second initial scene matrix using the second fusion module;

[0028] The sample object fusion vector is obtained by using the intermediate coding vector and the second output feature vector.

[0029] According to an embodiment of the present disclosure, the above-mentioned sample object sequence also corresponds to sample object label information.

[0030] According to an embodiment of the present disclosure, adjusting the initial scene template according to the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information includes:

[0031] Determining sample object prediction information based on the sample object fusion vector;

[0032] Obtaining a loss function value based on the sample object prediction information and the sample object label information;

[0033] While keeping the model parameters of the feature extraction module, the first encoder, and the first decoder unchanged, adjusting the first fusion module and the second fusion module according to the loss function value until a predetermined termination condition is satisfied, thereby obtaining the object prediction model; and

[0034] The initial scene template corresponding to the object prediction model is determined as the target scene template.

[0035] According to an embodiment of the present disclosure, the above-mentioned pre-training model also includes a multi-layer perception module, which includes M hidden layers. Each of the above-mentioned M hidden layers corresponds to N neurons, and M and N are both positive integers.

[0036] According to an embodiment of the present disclosure, constructing an initial scene template corresponding to the sample scene label information according to the sample scene label information corresponding to the sample object sequence includes:

[0037] Determining a target length value corresponding to the sample scene label information according to the sample scene label information;

[0038] Determine a target array according to the M hidden layers and the N neurons corresponding to each of the M hidden layers; and

[0039] The initial scene template is constructed according to the target length value and the target array.

[0040] According to an embodiment of the present disclosure, constructing the initial scene template according to the target length value and the target array includes:

[0041] Constructing a scene template to be processed corresponding to the sample scene label information according to the target length value and the target array; and

[0042] The multi-layer perception module is used to perform multi-layer perception processing on the scene template to be processed to obtain the initial scene template.

[0043] According to an embodiment of the present disclosure, the above-mentioned pre-trained model is trained in the following manner:

[0044] According to P pieces of sample object information, construct object data sets corresponding to each of the P pieces of sample object information, where P is a positive integer;

[0045] According to the Q sample scene information, construct a scene data set corresponding to each of the Q sample scene information, where Q is a positive integer;

[0046] Constructing the at least one pre-trained object sequence based on the object datasets corresponding to the P pieces of sample object information and the scene datasets corresponding to the Q pieces of sample scene information; and

[0047] The deep learning model is trained using the at least one pre-trained object sequence to obtain the pre-trained model.

[0048] According to an embodiment of the present disclosure, constructing the at least one pre-trained object sequence based on the object datasets corresponding to the P pieces of sample object information and the scene datasets corresponding to the Q pieces of sample scene information includes:

[0049] Constructing at least one candidate pre-trained object sequence based on the object datasets corresponding to the P pieces of sample object information and the scene datasets corresponding to the Q pieces of sample scene information;

[0050] For each candidate pre-trained object sequence in the at least one candidate pre-trained object sequence, dividing the candidate pre-trained object sequence based on a sliding window method to obtain a plurality of pre-trained object data; and

[0051] The pre-training object sequence is determined based on the plurality of pre-training object data.

[0052] According to another aspect of the present disclosure, there is provided an object prediction method, comprising:

[0053] Extract features from the target object sequence to obtain the target object feature vector;

[0054] According to the target scene label information corresponding to the target object sequence, a target scene template corresponding to the target scene label information is obtained from a scene template library;

[0055] Determining a target object fusion vector based on the target object feature vector and the target scene template; and

[0056] Determining an object prediction result corresponding to the target object sequence according to the target object fusion vector;

[0057] The above-mentioned scene template library is constructed using the above-mentioned scene template library construction method.

[0058] According to another aspect of the present disclosure, a device for constructing a scene template library is provided, comprising:

[0059] A first feature extraction module is used to extract features from a sample object sequence to obtain a sample object feature vector;

[0060] A first construction module is configured to construct an initial scene template corresponding to the sample scene label information according to the sample scene label information corresponding to the sample object sequence;

[0061] A first determining module is used to determine a sample object fusion vector based on the sample object feature vector and the initial scene template;

[0062] an adjustment module, configured to adjust the initial scene template according to the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information; and

[0063] The second construction module is configured to construct the scene template library according to the sample scene label information and the target scene template corresponding to the sample scene label information.

[0064] According to another aspect of the present disclosure, there is provided an object prediction device, comprising:

[0065] The second feature extraction module is used to extract features from the target object sequence to obtain a target object feature vector;

[0066] An acquisition module is used to acquire a target scene template corresponding to the target scene label information from a scene template library according to the target scene label information corresponding to the target object sequence;

[0067] A second determining module is configured to determine a target object fusion vector based on the target object sequence and the target scene template; and

[0068] A third determination module is used to determine the object prediction result corresponding to the target object sequence according to the target object fusion vector;

[0069] The above-mentioned scene template library is constructed using the above-mentioned scene template library construction device.

[0070] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0071] one or more processors;

[0072] a memory for storing one or more instructions,

[0073] When the one or more instructions are executed by the one or more processors, the one or more processors implement the method described in the present disclosure.

[0074] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which executable instructions are stored. When the executable instructions are executed by a processor, the processor implements the method described in the present disclosure.

[0075] According to another aspect of the present disclosure, a computer program product is provided. The computer program product includes computer-executable instructions. When the computer-executable instructions are executed, they are used to implement the method described in the present disclosure.

[0076] According to the embodiments of the present disclosure, since the sample object fusion vector is determined based on the sample object feature vector and the initial scene template, the sample object feature vector is obtained by extracting features from the sample object sequence, and the initial scene template is constructed based on the sample scene label information corresponding to the sample object sequence, the initial scene template is adjusted based on the sample object fusion vector so that the obtained target scene template can adapt to the scene type represented by the sample scene label information. On this basis, since the scene template library is constructed based on the sample scene label information and the target scene template corresponding to the sample scene label information, the scene template library can maintain target scene templates under different scene types, at least partially overcoming the technical problem in the related art that a single model cannot meet the needs of time series prediction based on time series data under different scenarios, improving the construction efficiency of the scene template library, and facilitating the subsequent acquisition of sample scene label information corresponding to the scene type from the scene template library according to the scene type. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0078] Figure 1 The system architecture of the method for constructing a scene template library and the object prediction method according to the embodiments of the present disclosure is schematically shown;

[0079] Figure 2 A flowchart schematically illustrates a method for constructing a scene template library according to an embodiment of the present disclosure;

[0080] Figure 3AAn example diagram schematically illustrates a training process of a pre-trained model according to an embodiment of the present disclosure;

[0081] Figure 3B Schematically illustrates an example schematic diagram of a process of dividing a candidate pre-training object sequence to obtain multiple pre-training object data based on a sliding window method according to an embodiment of the present disclosure;

[0082] Figure 4 Schematically illustrates an example of a process of constructing an initial scene template corresponding to sample scene label information according to sample scene label information corresponding to a sample object sequence according to an embodiment of the present disclosure;

[0083] Figure 5 Schematically illustrates an example schematic diagram of a process for determining a sample object fusion vector according to an embodiment of the present disclosure;

[0084] Figure 6 An example diagram schematically illustrates a training process of an object prediction model according to an embodiment of the present disclosure;

[0085] Figure 7 Schematically shows a flow chart of an object prediction method according to an embodiment of the present disclosure;

[0086] Figure 8 A block diagram schematically illustrates a device for constructing a scene template library according to an embodiment of the present disclosure;

[0087] Figure 9 A block diagram schematically illustrates an object prediction device according to an embodiment of the present disclosure; and

[0088] Figure 10 A block diagram of an electronic device suitable for implementing a method for constructing a scene template library and an object prediction method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0089] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0090] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0091] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0092] When expressions such as “at least one of A, B, and C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, and C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.). When expressions such as “at least one of A, B, or C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, or C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.).

[0093] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of the data involved (for example, including but not limited to user personal information) all comply with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken with respect to user personal information to prevent unauthorized access to user personal information data and to maintain the security of user personal information, network security, and national security.

[0094] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0095] For example, after collecting the target object sequence, your information can be desensitized using methods including de-identification or anonymization to protect the security of your information.

[0096] A time series can refer to a sequence of values ​​of the same statistical indicator arranged in chronological order. Time series forecasting methods can include at least one of the following: a time series forecasting method based on a statistical model, a time series forecasting method based on a machine learning model, and a time series forecasting method based on a deep learning model.

[0097] Since the models used for time series prediction need to be trained for specific, single scenarios, and the characteristics of each scenario are very different, a single model is difficult to solve the problems of all scenarios, so the transferability between models is poor.

[0098] In addition, since the model form used for time series prediction is relatively fixed, the model needs to be retrained when the feature parameters change, resulting in poor model flexibility.

[0099] In order to at least partially solve the technical problems existing in the related art, the present disclosure provides a method for constructing a scene template library, an object prediction method and device, and an electronic device, which can be applied to the fields of computer technology, big data technology, and artificial intelligence technology. The method for constructing the scene template library includes: extracting features from a sample object sequence to obtain a sample object feature vector; constructing an initial scene template corresponding to the sample scene label information based on sample scene label information corresponding to the sample object sequence; determining a sample object fusion vector based on the sample object feature vector and the initial scene template; adjusting the initial scene template based on the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information; and constructing a scene template library based on the sample scene label information and the target scene template corresponding to the sample scene label information.

[0100] Figure 1 The schematic diagram shows the system architecture of the method for constructing a scene template library and the object prediction method according to the embodiment of the present disclosure. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0101] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0102] The user may use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0103] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0104] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received user requests and other data, and feed back processing results (e.g., web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0105] It should be noted that the scene template library construction method and object prediction method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the scene template library construction device and object prediction device provided in the embodiment of the present disclosure can generally be set in the server 105. The scene template library construction method and object prediction method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the scene template library construction device and object prediction device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0106] Alternatively, the scene template library construction method and object prediction method provided in the embodiments of the present disclosure may also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the scene template library construction device and object prediction device provided in the embodiments of the present disclosure may also be set in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0107] It should be understood that Figure 1 The number of the first terminal device, the second terminal device, the third terminal device, the network and the server is only illustrative. According to implementation requirements, there can be any number of the first terminal device, the second terminal device, the third terminal device, the network and the server.

[0108] It should be noted that the sequence numbers of the operations in the following method are only used to indicate the operation for the purpose of description, and should not be regarded as indicating the order in which the operations should be performed. Unless explicitly stated, the method does not need to be performed in the order shown.

[0109] Figure 2 The flowchart of the method for constructing a scene template library according to an embodiment of the present disclosure is schematically shown.

[0110] like Figure 2 As shown, the method 200 for constructing the scene template library includes operations S210 to S250.

[0111] In operation S210 , feature extraction is performed on the sample object sequence to obtain a sample object feature vector.

[0112] In operation S220 , an initial scene template corresponding to the sample scene label information is constructed according to the sample scene label information corresponding to the sample object sequence.

[0113] In operation S230 , a sample object fusion vector is determined according to the sample object feature vector and the initial scene template.

[0114] In operation S240 , the initial scene template is adjusted according to the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information.

[0115] In operation S250 , a scene template library is constructed based on the sample scene label information and the target scene template corresponding to the sample scene label information.

[0116] According to an embodiment of the present disclosure, a sample object sequence may include time series data corresponding to the sample objects. The sample objects may be associated with sample scenarios. The sample objects may have corresponding sample object label information. The sample object label information may be used to characterize a specific task type. The sample scenarios may have corresponding sample scenario label information. The sample scenario label information may be used to characterize a specific scenario type. Time series data may refer to data arranged in a certain time sequence, reflecting the values ​​of the same phenomenon at different time points. Future data trends can be predicted by analyzing time series data.

[0117] According to an embodiment of the present disclosure, a first predetermined model can be pre-trained using multiple pre-trained object sequences to obtain a pre-trained model with general time series prediction capabilities. The multiple pre-trained object sequences can each correspond to a different task type and scenario type. After obtaining a sample object sequence, the sample object sequence can be processed using a pre-trained model to perform time series prediction on the sample object sequence and obtain a sample object prediction result. The first predetermined model may include a model that can realize time series prediction of the sample object sequence. The first predetermined model may include at least one of the following: a first predetermined model based on a statistical model, a first predetermined model based on a machine learning model, and a first predetermined model based on a deep learning model.

[0118] For example, the first predetermined model based on the statistical model may include at least one of: a moving average model (MA), an exponential smoothing model (MS), and an ARIMA (Autoregressive Integrated Moving Average Model) model. The first predetermined model based on the machine learning model may include at least one of: a decision tree (DT) model, a gradient boosting decision tree (GBDT) model, and a support vector machine (SVM). Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and a transformer.

[0119] According to an embodiment of the present disclosure, after obtaining a sample object sequence, a second predetermined model may be used to process the sample object sequence to obtain a sample object feature vector. The second predetermined model may include a deep learning model capable of extracting features from the sample object sequence. The second predetermined model may include at least one of: a convolutional neural network, a recurrent neural network, and a transformer. The training method of the second predetermined model can be configured based on actual business needs and is not limited here. For example, the training method may include at least one of: unsupervised training, supervised training, and semi-supervised training.

[0120] According to an embodiment of the present disclosure, after obtaining the sample object feature vector, an initial scene template can be constructed based on the sample scene label information corresponding to the sample object sequence. For example, the length value can be determined based on the sample scene label information. An array is determined based on the number of hidden layers in the pre-trained model and the number of neurons corresponding to each hidden layer. An initial scene template is constructed based on the length value and the array. The initial scene template (i.e., Prompt) can include a parameter matrix related to the scene type. The initial scene template can be combined with the sample object sequence to give the pre-trained model context to guide the task that the pre-trained model should perform next, that is, the downstream task can be transformed based on the pre-trained model.

[0121] According to an embodiment of the present disclosure, after obtaining the initial scene template, a third predetermined model can be used to process the sample object feature vector and the initial scene template to obtain a sample object fusion vector. The third predetermined model may include a deep learning model capable of achieving feature fusion. The third predetermined model may include at least one of: a transformer, a feature pyramid network (FPN), an LSTM (Long Short-Term Memory), and a Bi-LSTM (Bi-directional Long Short-Term Memory).

[0122] For example, when the third predetermined model is a converter, the second encoder in the converter can be used to process the sample object feature vector and the initial scene template to obtain an encoding vector, and the second decoder in the converter can be used to process the encoding vector to obtain a sample object fusion vector.

[0123] According to an embodiment of the present disclosure, after obtaining a sample object fusion vector, the initial scene template can be adjusted based on the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information. For example, an object prediction model can be obtained based on a pre-trained model combined with the initial scene template, and the object prediction model can be used to process the sample object sequence to obtain the sample object fusion vector. Based on the sample object fusion vector, the sample object prediction information is determined. While maintaining the model parameters of the pre-trained model unchanged, the initial scene template is adjusted based on the sample object prediction information and the sample object label information to obtain a target scene template corresponding to the sample scene label information.

[0124] According to an embodiment of the present disclosure, after obtaining a target scene template, a scene template library can be constructed based on sample scene tag information and the target scene template corresponding to the sample scene tag information. The scene template library can be used to store the sample scene tag information and the target scene template. The scene template library can maintain at least one key-value relationship. Each of the at least one key-value relationship can include key information of the sample scene tag information and value information of the target scene template.

[0125] According to the embodiments of the present disclosure, since the sample object fusion vector is determined based on the sample object feature vector and the initial scene template, the sample object feature vector is obtained by extracting features from the sample object sequence, and the initial scene template is constructed based on the sample scene label information corresponding to the sample object sequence, the initial scene template is adjusted based on the sample object fusion vector so that the obtained target scene template can adapt to the scene type represented by the sample scene label information. On this basis, since the scene template library is constructed based on the sample scene label information and the target scene template corresponding to the sample scene label information, the scene template library can maintain target scene templates under different scene types, at least partially overcoming the technical problem in the related art that a single model cannot meet the needs of time series prediction based on time series data under different scenarios, improving the construction efficiency of the scene template library, and facilitating the subsequent acquisition of sample scene label information corresponding to the scene type from the scene template library according to the scene type.

[0126] Reference below Figure 3A 、 Figure 3B 、 Figure 4 、 Figure 5 and Figure 6 , the method 200 for constructing a scene template library according to an embodiment of the present invention is further described.

[0127] According to an embodiment of the present disclosure, the pre-trained model can be trained in the following manner.

[0128] Based on P pieces of sample object information, an object dataset corresponding to each of the P pieces of sample object information is constructed, where P is a positive integer. Based on Q pieces of sample scene information, a scene dataset corresponding to each of the Q pieces of sample scene information is constructed, where Q is a positive integer. Based on the object dataset corresponding to each of the P pieces of sample object information and the scene dataset corresponding to each of the Q pieces of sample scene information, at least one pre-trained object sequence is constructed. A deep learning model is trained using the at least one pre-trained object sequence to obtain a pre-trained model.

[0129] According to embodiments of the present disclosure, sample object information may include the time step length of historical input data and the time step length of predicted output data. Based on the time step length of historical input data and the time step length of predicted output data corresponding to each of the P pieces of sample object information, an object dataset corresponding to each of the P pieces of sample object information may be constructed. Sample scene information may include the length of a scene dataset. Based on the length of the scene dataset corresponding to each of the Q pieces of sample scene information, a scene dataset corresponding to each of the Q pieces of sample scene information may be constructed.

[0130] According to an embodiment of the present disclosure, after obtaining an object dataset corresponding to each of P sample object information and a scene dataset corresponding to each of Q sample scene information, P×Q pre-trained object sequences can be constructed. Each pre-trained object sequence can have a scene dataset to construct an object dataset to determine the time step length of the historical input data and the time step length of the predicted output data in each training task. Taking the dimension of the historical input data as (k+1)×1 and the size of the predicted output data as n×1 as an example, the pre-trained object sequence can be as shown in Table 1 below.

[0131] Table 1

[0132]

[0133] According to an embodiment of the present disclosure, a deep learning model can be trained based on a first loss function according to at least one pre-trained object sequence and sample object label information corresponding to each of the at least one pre-trained object sequence to obtain a pre-trained model. The first loss function can include at least one of the following: Huber Loss Function, Mean Squared Error Loss Function (MSE), Mean Absolute Error Loss Function (MAE), Regression Loss Function, and Hinge Loss Function.

[0134] According to an embodiment of the present disclosure, taking the Huber loss function and δ as a hyperparameter as an example, the Huber loss function can combine the advantages of the mean square error loss function and the mean absolute error loss function to improve robustness to outliers. For example, when the Huber loss is between [0-δ, 0+δ], the Huber loss function is equivalent to the mean square error loss function. When the Huber loss is between [-∞, δ] and [δ, +∞], the Huber loss function is equivalent to the mean absolute error loss function.

[0135] According to the embodiments of the present disclosure, since the pre-trained object sequence is constructed based on the object data set and the scene data set, the deep learning model can be trained through pre-trained object sequences of different scenes and different formats while ensuring data quality, thereby improving the training efficiency and accuracy of the deep learning model, and further improving the generalization ability of the obtained pre-trained model, so that the obtained pre-trained model can have a general time series prediction capability for different scene types and different task types, so as to ensure the scalability and extensibility of subsequent pre-trained models to adapt to the scene types.

[0136] Figure 3A An example diagram schematically illustrates the training process of a pre-trained model according to an embodiment of the present disclosure.

[0137] like Figure 3A As shown in 300A, object datasets 302 corresponding to the P pieces of sample object information 301 can be constructed based on P pieces of sample object information 301. Scene datasets 304 corresponding to the Q pieces of sample scene information 303 can be constructed based on Q pieces of sample scene information 303.

[0138] At least one pre-training object sequence 305 is constructed based on the object dataset 302 corresponding to each of the P sample object information 301 and the scene dataset 304 corresponding to each of the Q sample scene information 303.

[0139] At least one pre-trained object sequence 305 is input into a deep learning model 306, which outputs a pre-trained object prediction result 307. The pre-trained object prediction result 307 and sample object label information 308 are input into a first loss function 309, which outputs a first loss function value 310. The first loss function value 310 is used to adjust the model parameters of the deep learning model 306 to obtain a pre-trained model.

[0140] According to an embodiment of the present disclosure, constructing at least one pre-trained object sequence based on object datasets corresponding to P pieces of sample object information and scene datasets corresponding to Q pieces of sample scene information may include the following operations.

[0141] At least one candidate pre-trained object sequence is constructed based on object datasets corresponding to each of the P sample object information and scene datasets corresponding to each of the Q sample scene information. For each candidate pre-trained object sequence in the at least one candidate pre-trained object sequence, the candidate pre-trained object sequence is partitioned based on a sliding window method to obtain a plurality of pre-trained object data. A pre-trained object sequence is determined based on the plurality of pre-trained object data.

[0142] According to an embodiment of the present disclosure, during the training process of the pre-trained model, a P×Q candidate pre-trained object sequence can be constructed based on an object dataset corresponding to each of the P sample object label information and a scene dataset corresponding to each of the Q sample scene label information. At least one candidate pre-trained object sequence is extracted from the P×Q candidate pre-trained object sequence without replacement and trained until all pre-trained object sequences are trained. Based on a sliding window method, the candidate pre-trained object sequence can be partitioned to continuously construct pre-trained object data.

[0143] Figure 3B The figure schematically shows an example process of dividing a candidate pre-training object sequence based on a sliding window method to obtain multiple pre-training object data according to an embodiment of the present disclosure.

[0144] like Figure 3B As shown, taking the sliding window 311 having a length of k+n+1 and a sliding step size of 1 as an example, in 300B, the plurality of pre-training object data may include values ​​from the time step tk-1 to the time step t+n. k, n, and t can be set according to actual business needs and are not limited here.

[0145] For example, if the first k+1 data are historical input data and the last n data are predicted target data, in this case, the pre-training object sequence may include values ​​from time step tk-1 to time step t. The sample object label information may include values ​​from time step t+1 to time step t+n.

[0146] According to an embodiment of the present disclosure, the pre-training model further includes a multi-layer perception module, which includes M hidden layers, each of the M hidden layers corresponding to N neurons, and M and N are both positive integers.

[0147] According to an embodiment of the present disclosure, operation S220 may include the following operations.

[0148] Based on the sample scene label information, a target length value corresponding to the sample scene label information is determined. Based on the M hidden layers and the N neurons corresponding to each of the M hidden layers, a target array is determined. Based on the target length value and the target array, an initial scene template is constructed.

[0149] According to an embodiment of the present disclosure, the initial scene template may be constructed as shown in the following formula (1).

[0150] P θ =|P idx |×dim(h) (1)

[0151] Among them, P θ Can characterize the initial scene template, P idx It can represent the target length value corresponding to the sample scene label information, h can represent the hidden layer dimension of the pre-trained model, and dim(h) can represent the target array.

[0152] According to an embodiment of the present disclosure, constructing an initial scene template according to a target length value and a target array may include the following operations.

[0153] According to the target length value and the target array, a scene template to be processed corresponding to the sample scene label information is constructed. The multi-layer perception module is used to perform multi-layer perception processing on the scene template to obtain the initial scene template.

[0154] According to an embodiment of the present disclosure, the scene template to be processed and the initial scene template may have the same length in the first dimension, but different lengths in the second dimension. The method of performing multi-layer perception processing on the scene template to be processed using the multi-layer perception module to obtain the initial scene template can be shown in the following formula (2).

[0155] P θ [i,:] = MLP(P′ θ [i,:]) (2)

[0156] Among them, P θ The initial scene template, P′, can be represented θ It can represent the scene template to be processed, MLP() can represent the processing using the multi-layer perception module, and i can represent the time step index of the initial scene template.

[0157] According to the embodiments of the present disclosure, since the scene template to be processed is constructed based on the target length value and the target array, it can fully utilize the scene data, improve the correlation between the scene template to be processed and the sample scene label information, and thus improve the quality of the scene template to be processed. On this basis, by using the multi-layer perception module to perform multi-layer perception processing on the scene template to be processed, a more accurate initial scene template that meets the actual scene requirements can be obtained, improving the efficiency and accuracy of the automatic construction of the initial scene template.

[0158] Figure 4The following schematically illustrates an example process of constructing an initial scene template corresponding to sample scene label information according to sample scene label information corresponding to a sample object sequence according to an embodiment of the present disclosure.

[0159] like Figure 4 As shown, in 400, a target length value 402 corresponding to the sample scene label information 401 can be determined according to the sample scene label information 401. A target array 405 can be determined according to M hidden layers 403 and N neurons 404 corresponding to each of the M hidden layers.

[0160] According to the target length value 402 and the target array 405, a scene template 406 to be processed corresponding to the sample scene label information is constructed. The scene template 406 to be processed is subjected to multi-layer perception processing by a multi-layer perception module to obtain an initial scene template 407.

[0161] According to an embodiment of the present disclosure, operation S230 may include the following operations.

[0162] The sample object feature vector and the first initial scene matrix are fused to obtain a first output feature vector. The first output feature vector is encoded to obtain an intermediate encoding vector. The sample object feature vector and the second initial scene matrix are fused to obtain a second output feature vector. The intermediate encoding vector and the second output feature vector are decoded to obtain a sample object fused vector.

[0163] According to an embodiment of the present disclosure, the initial scene template may include a first initial scene matrix and a second initial scene matrix.

[0164] According to an embodiment of the present disclosure, θ (P θ1 , P θ2 ) represents the initial scene template, where P θ1 The first initial scene matrix, P θ2 The second initial scene matrix can be represented. The parameters of the initial scene template can be randomly generated during fine-tuning training.

[0165] According to embodiments of the present disclosure, when fine-tuning a pre-trained model for a specific scene type, the initial scene template and the pre-trained model can be linked. For example, a first output feature vector can be determined based on a sample object feature vector and a first initial scene matrix. A second output feature vector can be determined based on the sample object feature vector and a second initial scene matrix. A sample object fusion vector can be determined based on the first output feature vector and the second output feature vector.

[0166] According to the embodiments of the present disclosure, by adopting vector encoding and vector decoding, the splicing of sample object feature vectors and initial scene templates can be realized in specific scene types, so that the obtained sample object fusion vector can fuse the object information corresponding to the sample object feature vector and the scene information corresponding to the initial scene template, thereby improving the accuracy and completeness of the sample object fusion vector.

[0167] Figure 5 An example schematic diagram of a process for determining a sample object fusion vector according to an embodiment of the present disclosure is schematically shown.

[0168] like Figure 5 As shown, in 500 , the sample object feature vector 501 and the first initial scene matrix 5021 may be fused to obtain a first output feature vector 503 . The first output feature vector 503 may be encoded to obtain an intermediate encoding vector 504 .

[0169] The sample object feature vector 501 and the second initial scene matrix 5022 are fused to obtain a second output feature vector 505 .

[0170] The intermediate coding vector 504 and the second output feature vector 505 are decoded to obtain a sample object fusion vector 506.

[0171] According to an embodiment of the present disclosure, operation S210 may include the following operations.

[0172] The sample object sequence is input into a feature extraction module in a pre-trained pre-training model to obtain a sample object feature vector, wherein the pre-training model further includes a first fusion module, a first encoder, a second fusion module and a first decoder.

[0173] According to an embodiment of the present disclosure, the first output feature vector may be obtained by processing the sample object feature vector and the first initial scene matrix using the first fusion module. The intermediate coding vector may be obtained by processing the first output feature vector using the first encoder. The second output feature vector may be obtained by processing the sample object feature vector and the second initial scene matrix using the second fusion module. The sample object fusion vector may be obtained using the intermediate coding vector and the second output feature vector.

[0174] According to an embodiment of the present disclosure, the sample object sequence X can be input into a feature extraction module (eg, an embedding layer) in a pre-trained model to obtain a sample object feature vector E. The dimension of the sample object feature vector E can be (k+1)×h.

[0175] According to an embodiment of the present disclosure, at the first encoder side, a first fusion module can be used to combine the sample object feature vector E and the first initial scene matrix Pθ1 Splicing is performed to obtain the first output feature vector [P θ1 , E]. The first output feature vector [P θ1 , E] as the input of the first encoder.

[0176] According to an embodiment of the present disclosure, a first encoder may include a first input layer and a first hidden layer. The first encoder may be configured to encode a first output feature vector. For example, the first output feature vector may be encoded using the first input layer of the first encoder to obtain an auxiliary encoding vector. The auxiliary encoding vector may be processed using the first hidden layer of the first encoder to obtain an intermediate encoding vector.

[0177] According to an embodiment of the present disclosure, at the first decoder side, a second fusion module can be used to combine the sample object feature vector E and the second initial scene matrix P θ2 Splicing is performed to obtain the second output feature vector [P θ2 , E]. The second output feature vector P θ2 , E serves as the input of the first decoder.

[0178] According to an embodiment of the present disclosure, the first decoder may include a second hidden layer and a first output layer. The first decoder may be configured to reconstruct the intermediate coding vector and the second output feature vector. For example, the second hidden layer of the first decoder may be used to decode the intermediate coding vector and the second output feature vector to obtain an auxiliary decoding vector. The auxiliary decoding vector may be processed using the first output layer of the first decoder to obtain a sample object fusion vector.

[0179] According to an embodiment of the present disclosure, the sample object sequence also corresponds to sample object label information.

[0180] According to an embodiment of the present disclosure, operation S240 may include the following operations.

[0181] Determine sample object prediction information based on the sample object fusion vector. Obtain a loss function value based on the sample object prediction information and the sample object label information. While maintaining the model parameters of the feature extraction module, the first encoder, and the first decoder unchanged, adjust the first fusion module and the second fusion module based on the loss function value until a predetermined termination condition is met, thereby obtaining an object prediction model. Determine the initial scene template corresponding to the object prediction model as the target scene template.

[0182] According to an embodiment of the present disclosure, a loss function value can be obtained based on the second loss function and the object prediction information and the sample object label information. During the training process, the model parameters of the pre-trained model can be frozen, that is, the model parameters of the pre-trained model are not updated. For example, the model parameters of the feature extraction module, the first encoder, and the first decoder can remain unchanged.

[0183] According to an embodiment of the present disclosure, the parameters of the first initial scene matrix and the second initial scene matrix can be updated according to the loss function value until a predetermined end condition is met. The predetermined end condition may include that the parameter values ​​of the initial scene template tend to be stable. For example, the model parameters of the first fusion module and the second fusion module can be adjusted according to the loss function value. In this case, the initial scene template can be adjusted to a target scene template with more significant scenario features to guide the pre-trained model to perform an object prediction model for object prediction under a specific scene type.

[0184] According to an embodiment of the present disclosure, by obtaining a loss function value based on sample object prediction information and sample object label information, and further adjusting the first initial scene matrix corresponding to the first fusion module and the second initial scene matrix corresponding to the second fusion module, the problems of high modeling cost and poor model transferability caused by the need to model different scenes separately are at least partially overcome, and the adaptability of the object prediction model to the scene type can be improved, thereby improving the generation efficiency and quality of the target scene template.

[0185] Figure 6 An example diagram schematically illustrates a training process of an object prediction model according to an embodiment of the present disclosure.

[0186] like Figure 6 As shown in 600 , the pre-trained model 600 may include a feature extraction module 600_1 , a first fusion module 600_2 , a first encoder 600_3 , a second fusion module 600_4 and a first decoder 600_5 .

[0187] The sample object sequence 601 may be input into the feature extraction module 600_1 , which outputs a sample object feature vector 602 .

[0188] The first fusion module 600_2 may fuse the first initial scene matrix 603_1 and the sample object feature vector 602 to output a first output feature vector 604. The first output feature vector 604 may be input to the first encoder 600_3 to output an intermediate encoding vector 605.

[0189] The second fusion module 600_4 may perform fusion processing on the second initial scene matrix 603_2 and the sample object feature vector 602 , and output a second output feature vector 606 .

[0190] The intermediate coding vector 605 and the second output feature vector 606 may be input to the first decoder 600_5, which outputs a sample object fusion vector 607. Based on the sample object fusion vector 607, sample object prediction information 608 is determined.

[0191] Based on the second loss function 609, a second loss function value 610 is obtained according to the sample object prediction information 608 and the sample object label information corresponding to the sample object sequence 601. The second loss function value 610 is used to adjust the model parameters of the first fusion module 600_2 (i.e., the first initial scene matrix 603_1) and the second fusion module 600_4 (i.e., the second initial scene matrix 603_2) in the pre-trained model 600 to obtain an object prediction model.

[0192] According to an embodiment of the present disclosure, operation S250 may include the following operations.

[0193] Determine the sample scene tag information as the scene template key information. Determine the target scene template as the scene template value information. Construct a scene template key-value relationship based on the scene template key information and the scene template value information. Store the scene template key-value relationship in the scene template library.

[0194] According to an embodiment of the present disclosure, a scene template library can be used to maintain at least one scene template key-value relationship. Each of the at least one scene template key-value relationship can include scene template key information and scene template value information. After completing the training of the object prediction model, the sample scene label information can be determined as the scene template key information (i.e., Key). The target scene template is determined as the scene template value information (i.e., Value).

[0195] According to the embodiments of the present disclosure, since the scene template key-value relationship is constructed based on the sample scene label information and the target scene template, the scene template library can be used to manage the sample scene label information and the target scene template, thereby improving the management efficiency and accuracy of the target scene template. By maintaining a general pre-trained model and scene template library in memory, when performing subsequent object prediction, the target scene template corresponding to the scene can be obtained from the scene template library, and the target scene template can be combined with the pre-trained model, thereby saving computing resources and time costs, and being able to stimulate the pre-trained model's prediction capabilities for different scene types and different task types, thereby improving the efficiency of scene type switching, and thereby improving the flexibility and efficiency of object prediction.

[0196] The above are merely exemplary embodiments, but are not limited thereto. Other scene template library construction methods known in the art may also be included, as long as the construction efficiency of the scene template library can be improved.

[0197] Figure 7 The flowchart of the object prediction method according to an embodiment of the present disclosure is schematically shown.

[0198] like Figure 7 As shown, the object prediction method 700 includes operations S710 to S740.

[0199] In operation S710 , feature extraction is performed on the target object sequence to obtain a target object feature vector.

[0200] In operation S720 , according to the target scene label information corresponding to the target object sequence, a target scene template corresponding to the target scene label information is acquired from a scene template library.

[0201] In operation S730 , a target object fusion vector is determined based on the target object feature vector and the target scene template.

[0202] In operation S740 , an object prediction result corresponding to the target object sequence is determined according to the target object fusion vector.

[0203] According to an embodiment of the present disclosure, the scene template library may be constructed using the scene template library construction method 200 .

[0204] According to the embodiments of the present disclosure, for the description of the target object sequence, target object feature vector, target scene label information and target object fusion vector, please refer to the relevant content above for the sample object sequence, sample object feature vector, sample scene label information and sample object fusion vector, which will not be repeated here.

[0205] According to an embodiment of the present disclosure, target scene label information can be determined based on the task type corresponding to the target object sequence. Based on the target scene label information, a target scene template corresponding to the target scene label information is obtained from a scene template library. The scene template library may include at least one sample scene label information and a target scene template corresponding to each of the at least one sample scene label information. For example, the target scene label information can be matched with the at least one sample scene label information to obtain the target scene template.

[0206] According to embodiments of the present disclosure, after obtaining a target scene template, the target scene template can be spliced ​​together based on a pre-trained model to perform object prediction for a scene type represented by the target scene label information, thereby obtaining a target object fusion vector. Based on the target object fusion vector, an object prediction result corresponding to the target object sequence can be determined.

[0207] According to the embodiments of the present disclosure, since the target scene template is obtained from the scene template library based on the target scene label information corresponding to the target object sequence, the target scene template can adapt to the scene type represented by the target scene label information. On this basis, since the target object fusion vector is determined based on the target object feature vector and the target scene template, and the target object feature vector is obtained by feature extraction of the target object sequence, by determining the object prediction result corresponding to the target object sequence based on the target object fusion vector, the scene adaptability and object prediction efficiency of the object prediction are improved.

[0208] The above are merely exemplary embodiments, but are not limited thereto, and may also include other object prediction methods known in the art, as long as they can improve the scene adaptability and object prediction efficiency of object prediction.

[0209] Figure 8 A block diagram schematically illustrates a device for constructing a scene template library according to an embodiment of the present disclosure.

[0210] like Figure 8 As shown, the scene template library construction device 800 may include a first feature extraction module 810 , a first construction module 820 , a first determination module 830 , an adjustment module 840 and a second construction module 850 .

[0211] The first feature extraction module 810 is configured to extract features from a sample object sequence to obtain a sample object feature vector.

[0212] The first construction module 820 is configured to construct an initial scene template corresponding to the sample scene label information according to the sample scene label information corresponding to the sample object sequence.

[0213] The first determining module 830 is configured to determine a sample object fusion vector according to the sample object feature vector and the initial scene template.

[0214] The adjustment module 840 is configured to adjust the initial scene template according to the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information.

[0215] The second constructing module 850 is configured to construct a scene template library according to the sample scene label information and the target scene template corresponding to the sample scene label information.

[0216] According to an embodiment of the present disclosure, the initial scene template includes a first initial scene matrix and a second initial scene matrix.

[0217] According to an embodiment of the present disclosure, the first determining module 830 may include a first fusion unit, an encoding unit, a second fusion unit, and a decoding unit.

[0218] The first fusion unit is used to fuse the sample object feature vector and the first initial scene matrix to obtain a first output feature vector.

[0219] The encoding unit is used to encode the first output feature vector to obtain an intermediate encoding vector.

[0220] The second fusion unit is used to fuse the sample object feature vector and the second initial scene matrix to obtain a second output feature vector.

[0221] The decoding unit is used to decode the intermediate coding vector and the second output feature vector to obtain a sample object fusion vector.

[0222] According to an embodiment of the present disclosure, the second construction module 850 may include a first determining unit, a second determining unit, a first construction unit, and a storage unit.

[0223] The first determining unit is configured to determine the sample scene label information as the scene template key information.

[0224] The second determining unit is configured to determine the target scene template as the scene template value information.

[0225] The first constructing unit is configured to construct a scene template key-value relationship according to the scene template key information and the scene template value information.

[0226] The storage unit is used to store the scene template key-value relationship in the scene template library.

[0227] According to an embodiment of the present disclosure, the first feature extraction module 810 may include an input unit.

[0228] The input unit is used to input the sample object sequence into the feature extraction module of the pre-trained pre-trained model to obtain the sample object feature vector, wherein the pre-trained model also includes a first fusion module, a first encoder, a second fusion module and a first decoder.

[0229] According to an embodiment of the present disclosure, the first output feature vector is obtained by processing the sample object feature vector and the first initial scene matrix using the first fusion module.

[0230] According to an embodiment of the present disclosure, the intermediate coding vector is obtained by processing the first output feature vector using the first encoder.

[0231] According to an embodiment of the present disclosure, the second output feature vector is obtained by processing the sample object feature vector and the second initial scene matrix using the second fusion module.

[0232] According to an embodiment of the present disclosure, the sample object fusion vector is obtained by using the intermediate coding vector and the second output feature vector.

[0233] According to an embodiment of the present disclosure, the sample object sequence also corresponds to sample object label information.

[0234] According to an embodiment of the present disclosure, the adjustment module 840 may include a third determining unit, an obtaining unit, an adjusting unit, and a fourth determining unit.

[0235] The third determining unit is configured to determine sample object prediction information according to the sample object fusion vector.

[0236] The acquisition unit is used to obtain a loss function value based on the sample object prediction information and the sample object label information.

[0237] The adjustment unit is used to adjust the first fusion module and the second fusion module according to the loss function value while keeping the model parameters of the feature extraction module, the first encoder and the first decoder unchanged, until the predetermined end condition is met to obtain the object prediction model.

[0238] The fourth determining unit is configured to determine the initial scene template corresponding to the object prediction model as the target scene template.

[0239] According to an embodiment of the present disclosure, the pre-training model further includes a multi-layer perception module, which includes M hidden layers, each of the M hidden layers corresponding to N neurons, and M and N are both positive integers.

[0240] According to an embodiment of the present disclosure, the first construction module 820 may include a fifth determining unit, a sixth determining unit, and a second construction unit.

[0241] The fifth determining unit is configured to determine, according to the sample scene label information, a target length value corresponding to the sample scene label information.

[0242] The sixth determining unit is configured to determine a target array according to the M hidden layers and the N neurons corresponding to each of the M hidden layers.

[0243] The second construction unit is used to construct an initial scene template according to the target length value and the target array.

[0244] According to an embodiment of the present disclosure, the second construction unit may include a first construction subunit and a processing subunit.

[0245] The first construction subunit is used to construct a scene template to be processed corresponding to the sample scene label information according to the target length value and the target array.

[0246] The processing subunit is used to perform multi-layer perception processing on the scene template to be processed using the multi-layer perception module to obtain an initial scene template.

[0247] According to an embodiment of the present disclosure, the pre-trained model can be trained in the following manner.

[0248] Based on P pieces of sample object information, an object dataset corresponding to each of the P pieces of sample object information is constructed, where P is a positive integer. Based on Q pieces of sample scene information, a scene dataset corresponding to each of the Q pieces of sample scene information is constructed, where Q is a positive integer. Based on the object dataset corresponding to each of the P pieces of sample object label information and the scene dataset corresponding to each of the Q pieces of sample scene label information, at least one pre-trained object sequence is constructed. A deep learning model is trained using the at least one pre-trained object sequence to obtain a pre-trained model.

[0249] According to an embodiment of the present disclosure, constructing at least one pre-trained object sequence based on object datasets corresponding to P pieces of sample object information and scene datasets corresponding to Q pieces of sample scene information can be used to perform the following operations.

[0250] At least one candidate pre-training object sequence is constructed based on the object data sets corresponding to the P pieces of sample object information and the scene data sets corresponding to the Q pieces of sample scene information.

[0251] For each candidate pre-training object sequence in at least one candidate pre-training object sequence, the candidate pre-training object sequence is divided based on a sliding window method to obtain a plurality of pre-training object data.

[0252] A pre-training object sequence is determined based on a plurality of pre-training object data.

[0253] Figure 9 The figure schematically shows a block diagram of an object prediction device according to an embodiment of the present disclosure.

[0254] like Figure 9 As shown, the object prediction apparatus 900 may include a second feature extraction module 910 , an acquisition module 920 , a second determination module 930 , and a third determination module 940 .

[0255] The second feature extraction module 910 is used to extract features from the target object sequence to obtain a target object feature vector.

[0256] The acquisition module 920 is configured to acquire, from a scene template library, a target scene template corresponding to the target scene label information according to the target scene label information corresponding to the target object sequence.

[0257] The second determining module 930 is configured to determine a target object fusion vector according to the target object sequence and the target scene template.

[0258] The third determination module 940 is configured to determine an object prediction result corresponding to the target object sequence according to the target object fusion vector.

[0259] According to an embodiment of the present disclosure, the scene template library is constructed using the scene template library construction device 800 .

[0260] According to the modules, submodules, units, and subunits of the embodiments of the present invention, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be split into multiple modules for implementation. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in an appropriate combination of any of them. Alternatively, according to the modules, submodules, units, and subunits of the embodiments of the present invention, one or more thereof can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is run.

[0261] For example, any multiple of the first feature extraction module 810, the first construction module 820, the first determination module 830, the adjustment module 840 and the second construction module 850 can be combined into one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first feature extraction module 810, the first construction module 820, the first determination module 830, the adjustment module 840 and the second construction module 850 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the first feature extraction module 810, the first construction module 820, the first determination module 830, the adjustment module 840 and the second construction module 850 can be at least partially implemented as a computer program module, which can perform the corresponding function when executed.

[0262] It should be noted that the construction device part of the scene template library in the embodiment of the present disclosure corresponds to the construction method part of the scene template library in the embodiment of the present disclosure. The description of the construction device part of the scene template library specifically refers to the construction method part of the scene template library, which will not be repeated here.

[0263] For example, any multiple of the second feature extraction module 910, the acquisition module 920, the second determination module 930, and the third determination module 940 can be combined into one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the second feature extraction module 910, the acquisition module 920, the second determination module 930, and the third determination module 940 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the second feature extraction module 910 , the acquisition module 920 , the second determination module 930 and the third determination module 940 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0264] It should be noted that the object prediction device part in the embodiment of the present disclosure corresponds to the object prediction method part in the embodiment of the present disclosure. The description of the object prediction device part specifically refers to the object prediction method part and will not be repeated here.

[0265] Figure 10 A block diagram of an electronic device suitable for implementing a method for constructing a scene template library and an object prediction method according to an embodiment of the present disclosure is schematically shown. Figure 10 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0266] like Figure 10As shown, the computer electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1009 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for executing different actions of the method flow according to an embodiment of the present disclosure.

[0267] Various programs and data required for the operation of the electronic device 1000 are stored in the RAM 1003. The processor 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 1002 and / or the RAM 1003. It should be noted that the programs may also be stored in one or more memories other than the ROM 1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0268] According to an embodiment of the present disclosure, electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to bus 1004. Electronic device 1000 may further include one or more of the following components connected to I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card or a modem. Communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in drive 1010 as needed, so that computer programs read therefrom can be installed into storage section 1008 as needed.

[0269] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0270] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0271] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0272] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 1002 and / or the RAM 1003 described above and / or one or more memories other than the ROM 1002 and the RAM 1003 .

[0273] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the scene template library construction method and object prediction method provided by the embodiment of the present disclosure.

[0274] When the computer program is executed by the processor 1001, the above functions defined in the system / device of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0275] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0276] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0277] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in various ways, and all such combinations and / or couplings fall within the scope of the present disclosure.

[0278] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for constructing a scene template library, comprising: Perform feature extraction on the sample object sequence to obtain the sample object feature vector; Constructing, based on the sample scene label information corresponding to the sample object sequence, an initial scene template corresponding to the sample scene label information, wherein the initial scene template is constructed based on a length value and an array, the array is determined based on the number of hidden layers in a pre-trained model and the number of neurons corresponding to each hidden layer, the length value is determined based on the sample scene label information, and the initial scene template includes a parameter matrix related to the scene type; Determining a sample object fusion vector according to the sample object feature vector and the initial scene template; Adjusting the initial scene template according to the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information; as well as The scene template library is constructed according to the sample scene label information and the target scene template corresponding to the sample scene label information.

2. The method according to claim 1, wherein The initial scene template includes a first initial scene matrix and a second initial scene matrix; The determining of the sample object fusion vector according to the sample object feature vector and the initial scene template includes: Fusing the sample object feature vector and the first initial scene matrix to obtain a first output feature vector; performing encoding processing on the first output feature vector to obtain an intermediate encoding vector; fusing the sample object feature vector and the second initial scene matrix to obtain a second output feature vector; and The intermediate coding vector and the second output feature vector are decoded to obtain the sample object fusion vector.

3. The method according to claim 1, wherein The constructing the scene template library according to the sample scene label information and the target scene template corresponding to the sample scene label information includes: Determining the sample scene label information as scene template key information; Determine the target scene template as scene template value information; Constructing a scene template key-value relationship according to the scene template key information and the scene template value information; and The scene template key-value relationship is stored in the scene template library.

4. The method according to claim 2, wherein: Extracting features from the sample object sequence to obtain a sample object feature vector includes: Inputting the sample object sequence into a feature extraction module in a pre-trained pre-trained model to obtain the sample object feature vector, wherein the pre-trained model further includes a first fusion module, a first encoder, a second fusion module and a first decoder; The first output feature vector is obtained by processing the sample object feature vector and the first initial scene matrix using the first fusion module; The intermediate encoding vector is obtained by processing the first output feature vector using the first encoder; The second output feature vector is obtained by processing the sample object feature vector and the second initial scene matrix using the second fusion module; The sample object fusion vector is obtained by using the intermediate coding vector and the second output feature vector.

5. The method according to claim 4, wherein The sample object sequence also corresponds to sample object label information; The adjusting the initial scene template according to the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information includes: Determining sample object prediction information according to the sample object fusion vector; Obtaining a loss function value according to the sample object prediction information and the sample object label information; While keeping the model parameters of the feature extraction module, the first encoder, and the first decoder unchanged, adjusting the first fusion module and the second fusion module according to the loss function value until a predetermined end condition is met, thereby obtaining an object prediction model; and An initial scene template corresponding to the object prediction model is determined as the target scene template.

6. The method according to claim 4, wherein: The pre-training model further includes a multi-layer perception module, wherein the multi-layer perception module includes M hidden layers, each of the M hidden layers corresponds to N neurons, and M and N are both positive integers; The constructing, based on the sample scene label information corresponding to the sample object sequence, an initial scene template corresponding to the sample scene label information comprises: Determining, according to the sample scene label information, a target length value corresponding to the sample scene label information; Determining a target array according to the M hidden layers and the N neurons corresponding to each of the M hidden layers; and The initial scene template is constructed according to the target length value and the target array.

7. The method according to claim 6, wherein: The constructing the initial scene template according to the target length value and the target array includes: Constructing a scene template to be processed corresponding to the sample scene label information according to the target length value and the target array; and The multi-layer perception module is used to perform multi-layer perception processing on the scene template to be processed to obtain the initial scene template.

8. The method according to claim 4, wherein: The pre-trained model is trained in the following way: According to P pieces of sample object information, construct object data sets corresponding to each of the P pieces of sample object information, where P is a positive integer; According to Q pieces of sample scene information, construct a scene data set corresponding to each of the Q pieces of sample scene information, where Q is a positive integer; Constructing at least one pre-trained object sequence based on the object datasets corresponding to the P pieces of sample object information and the scene datasets corresponding to the Q pieces of sample scene information; and The deep learning model is trained using the at least one pre-trained object sequence to obtain the pre-trained model.

9. The method according to claim 8, wherein The constructing the at least one pre-trained object sequence according to the object datasets corresponding to the P pieces of sample object information and the scene datasets corresponding to the Q pieces of sample scene information comprises: Constructing at least one candidate pre-trained object sequence based on an object dataset corresponding to each of the P pieces of sample object information and a scene dataset corresponding to each of the Q pieces of sample scene information; For each candidate pre-trained object sequence in the at least one candidate pre-trained object sequence, Based on a sliding window method, the candidate pre-training object sequence is divided to obtain a plurality of pre-training object data; and The pre-training object sequence is determined according to the plurality of pre-training object data.

10. An object prediction method, comprising: Extract features from the target object sequence to obtain the target object feature vector; According to the target scene label information corresponding to the target object sequence, obtaining a target scene template corresponding to the target scene label information from a scene template library; Determining a target object fusion vector according to the target object feature vector and the target scene template; as well as Determining an object prediction result corresponding to the target object sequence according to the target object fusion vector; Wherein, the scene template library is constructed using the method described in any one of claims 1 to 9.

11. A device for constructing a scene template library, comprising: A first feature extraction module is used to extract features from a sample object sequence to obtain a sample object feature vector; a first construction module, configured to construct, based on sample scene label information corresponding to the sample object sequence, an initial scene template corresponding to the sample scene label information, wherein the initial scene template is constructed based on a length value and an array, the array being determined based on the number of hidden layers in a pre-trained model and the number of neurons corresponding to each hidden layer, the length being determined based on the sample scene label information, and the initial scene template including a parameter matrix related to the scene type; A first determining module, configured to determine a sample object fusion vector based on the sample object feature vector and the initial scene template; an adjustment module, configured to adjust the initial scene template according to the sample object fusion vector to obtain a target scene template corresponding to the sample scene label information; and The second construction module is configured to construct the scene template library according to the sample scene label information and the target scene template corresponding to the sample scene label information.

12. An object prediction device, comprising: The second feature extraction module is used to extract features from the target object sequence to obtain a target object feature vector; An acquisition module, configured to acquire, from a scene template library, a target scene template corresponding to the target scene label information according to the target scene label information corresponding to the target object sequence; A second determining module is used to determine a target object fusion vector according to the target object sequence and the target scene template; as well as A third determination module is configured to determine an object prediction result corresponding to the target object sequence according to the target object fusion vector; Wherein, the scene template library is constructed using the device described in claim 11.

13. An electronic device comprising: one or more processors; a memory for storing one or more instructions, Wherein, when the one or more instructions are executed by the one or more processors, the one or more processors are enabled to implement the method of any one of claims 1 to 9 or claim 10.

14. A computer-readable storage medium having executable instructions stored thereon, wherein when the executable instructions are executed by a processor, the processor is enabled to implement the method according to any one of claims 1 to 9 or claim 10.

15. A computer program product comprising computer executable instructions for implementing the method of any one of claims 1 to 9 or claim 10 when executed.

Images