Information pushing method and device, electronic equipment and storage medium

Abstract

The invention provides an information pushing method and device, electronic equipment and a storage medium, and relates to the technical field of finance, and the method comprises the steps: obtaining text features of a plurality of pieces of market text information of a candidate pushed product and visual features obtained through the fusion of visual sub-features of a plurality of pieces of visual data of the candidate pushed product, the visual sub-features are obtained by performing feature extraction on visual instructions corresponding to the visual data in a plurality of pre-trained visual instructions based on a task scene to which the visual data belongs in a plurality of preset task scenes by a pre-trained investment information pushing model, and the plurality of visual instructions are in one-to-one correspondence with the plurality of task scenes; and performing multi-modal feature fusion on the text features and the visual features, performing market prediction based on emotion analysis on the candidate push products according to the fused features so as to determine a push strategy for pushing the candidate push products to the target object, and executing investment information push according to the push strategy. According to the embodiment of the invention, more accurate investment information can be pushed to the target object.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to an information push method, device, electronic device and storage medium. Background Art

[0002] In related technologies, intelligent investment advisory systems can provide users with automated and personalized investment advice. Specifically, these intelligent investment advisory systems can use artificial intelligence technology to automatically perform real-time market detection and adjustments for each financial product in the investment portfolio provided to the user.

[0003] However, in the investment market, a financial product reflects its future market conditions through multimodal information or data. Since the intelligent investment advisory system in related technologies can only take into account single-modal data, the intelligent investment advisory system cannot actually take into account data in other modalities when predicting the market conditions of a product. This makes it impossible for the intelligent investment advisory system to accurately predict the market conditions of a product, and thus it is impossible to provide users with accurate investment advice. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose an information push method, device, electronic device and storage medium, aiming to push more accurate investment information to the target object.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application provides an information push method, comprising the following steps: Obtaining text features of multiple market text information of candidate push products and visual features of multiple visual data of the candidate push products, wherein the visual features are obtained by fusing visual sub-features of each visual data, and the visual sub-features are obtained by performing feature extraction based on a target visual instruction on the visual data using a pre-trained investment information push model, the target visual instruction being one of the multiple pre-trained visual instructions corresponding to a target task scenario, the target task scenario being one of the multiple preset task scenarios to which the visual data belongs, and a unique correspondence exists between one visual instruction and one task scenario; Performing multimodal feature fusion on the text features and the visual features to obtain fused features; Performing a market sentiment analysis based on the fusion features, and performing a market forecast for the candidate push product based on the market sentiment analysis results to obtain a market forecast result for the candidate push product; According to the market forecast result, a push strategy for pushing the candidate push product to the target object is determined, and investment information push is executed according to the push strategy.

[0006] In some embodiments, the method is performed by the investment information push model; The investment information push model is trained through the following steps: Obtaining a first sample set, wherein the first sample set includes a plurality of first sample pairs, each of the first sample pairs including a first training label and a visual instruction sample, each of the task scenarios corresponds to a plurality of the plurality of visual instruction samples, and the visual instruction samples of the plurality of first sample pairs in each task scenario are different from each other; Inputting the first sample pair into an initial model, and obtaining a first sample market prediction result through the initial model; performing a loss analysis on the visual instruction sample based on the first sample market prediction result and the first training label to obtain a first loss value, wherein the first loss value is used to adjust the parameters of the initial model and the visual instruction sample when a first preset training end condition is not met; When the first preset training end condition is met, the initial model after parameter adjustment is determined as the investment information push model, and the visual instructions for multiple task scenarios are determined based on multiple adjusted visual instruction samples.

[0007] In some embodiments, before obtaining the first sample set, the investment information push model is further trained through the following steps: Acquire a second sample set, wherein the second sample set includes a plurality of second sample pairs, the plurality of second sample pairs correspond to at least one task scene, the second sample pairs belonging to the same task scene correspond to the same visual instruction sample, and the second sample pairs include a second training label; Inputting one of the second sample pairs into the initial model, and obtaining a second sample market forecast result through the initial model; performing a loss analysis on multimodal feature fusion based on the second sample market prediction result and the second training label to obtain a second loss value, wherein the second loss value is used to adjust the parameters of the initial model when a second preset training end condition is not met; When the second preset training end condition is met, the input of the next second sample pair into the initial model is canceled, and the first sample set is obtained.

[0008] In some embodiments, the first sample market prediction result is obtained by the initial model under the first model parameters based on the first sample pair; The performing loss analysis on the visual instruction sample according to the first sample market prediction result and the first training label to obtain a first loss value includes: Based on the first sample market prediction result, calculating a first prediction probability of the push strategy sample determined by the initial model according to the first sample; The first loss value is calculated based on the first predicted probability and the first training label.

[0009] In some embodiments, the second sample market prediction result is obtained by the initial model under the second model parameters based on the second sample pair; The performing loss analysis on multimodal feature fusion according to the second sample market prediction result and the second training label to obtain a second loss value includes: Based on the second sample market prediction result, calculating a second prediction probability of the initial model for the push strategy sample determined according to the second sample; The second loss value is calculated based on the second predicted probability and the second training label.

[0010] In some embodiments, the text features are obtained by fusing text sub-features of multiple market text information, and the text sub-features are obtained by performing feature extraction on the market text information for target text instructions through the investment information push model. The target text instruction is one of the target information type corresponding to multiple pre-trained text instructions, and the target information type is one of the multiple preset information types to which the market text information belongs. There is a unique correspondence between one text instruction and one information type.

[0011] In some embodiments, the first sample pair further includes a first visual sample feature; After obtaining the first loss value, the training step of the investment information push model further includes: When the first preset training end condition is not met, determining, from the plurality of first sample pairs, a target first sample pair that belongs to the same task scenario as the current first sample pair and has not been input into the initial model; Based on the adjusted visual instruction sample in the current first sample pair, the visual instruction sample and the first visual sample features in the target first sample pair are updated, and the next first sample pair is input into the adjusted initial model.

[0012] To achieve the above-mentioned purpose, a second aspect of the embodiments of the present application provides an information push device, comprising: a feature acquisition module for acquiring text features of multiple market text information of candidate push products and visual features of multiple visual data of the candidate push products, wherein the visual features are obtained by fusing visual sub-features of each visual data, and the visual sub-features are obtained by performing feature extraction based on target visual instructions on the visual data using a pre-trained investment information push model, wherein the target visual instruction is one of the multiple pre-trained visual instructions corresponding to a target task scenario, and the target task scenario is one of the multiple preset task scenarios to which the visual data belongs, and there is a unique correspondence between one visual instruction and one task scenario; A multimodal feature fusion module, configured to perform multimodal feature fusion on the text features and the visual features to obtain fused features; A market prediction module is used to perform market sentiment analysis based on the fusion features, and to perform market prediction for the candidate push product based on the market sentiment analysis results to obtain a market prediction result for the candidate push product; The push execution module is used to determine a push strategy for pushing the candidate push products to the target object according to the market forecast result, and execute investment information push according to the push strategy.

[0013] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0014] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect.

[0015] The present application proposes an information push method, device, electronic device, and storage medium. The method can first obtain text features of multiple market text information of candidate push products and visual features of multiple visual data of the candidate push products, wherein the visual features are obtained by fusing visual sub-features of each visual data, and the visual sub-features are obtained by performing feature extraction based on target visual instructions on the visual data using a pre-trained investment information push model. The target visual instruction is one of the multiple pre-trained visual instructions corresponding to a target task scenario, and the target task scenario is one of the multiple preset task scenarios to which the visual data belongs. There is a unique correspondence between one visual instruction and one task scenario. Then, multimodal feature fusion is performed on the text features and the visual features to obtain fused features. Then, market sentiment analysis is performed based on the fused features. Based on the market sentiment analysis results, a market forecast of the candidate push product is performed to obtain a market forecast result of the candidate push product. Then, based on the market forecast result, a push strategy for pushing the candidate push product to the target object is determined, and investment information push is executed according to the push strategy. Since each visual sub-feature is extracted based on the target visual instructions corresponding to the target task scenario to which the visual data belongs, the investment information push model can pay attention to the detailed information in the visual data of different task scenarios, so that the extracted visual sub-features can be more accurate. In this way, when performing market sentiment analysis based on the fusion features, the investment information push model can make market forecasts based on market sentiment based on text features and visual features, while also paying attention to the impact of detailed information in the visual data of different task scenarios on the analysis of market sentiment, thereby making the market forecast results obtained by the investment information push model more accurate, which is conducive to improving the accuracy of the push strategy, and further conducive to pushing more accurate investment information about candidate push products to the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of an information push method provided by an embodiment of the present application; Figure 2 yes Figure 1 A schematic diagram of an embodiment of a training step of the investment information push model disclosed in; Figure 3 yes Figure 1 A schematic diagram of another training step embodiment of the investment information push model disclosed in; Figure 4 yes Figure 2 A flowchart of an embodiment of a sub-step of step 230; Figure 5 yes Figure 3 A flowchart of an embodiment of a sub-step of step 330; Figure 6 yes Figure 1 A schematic diagram of another training step embodiment of the investment information push model disclosed in; Figure 7 This is a schematic diagram of the structure of the information push device provided in an embodiment of the present application; Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0018] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0020] First, let’s analyze some of the terms used in this application: Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0021] In the related art, an intelligent investment advisory system can be used to provide users with automated and personalized investment advice. However, in the investment market, a financial product reflects its future market conditions through multimodal information or data. Since the intelligent investment advisory system in the related art can only take into account a single modal data, the intelligent investment advisory system cannot actually take into account data in other modalities when predicting the market conditions of a product. This makes it impossible for the intelligent investment advisory system to accurately predict the market conditions of a product, and thus it is impossible to provide users with an accurate investment advice.

[0022] In order to push more accurate investment information to a target object, embodiments of the present application provide an information push method, an information push device, an electronic device, and a computer-readable storage medium. The method can first obtain text features of multiple market text information of candidate push products and visual features of multiple visual data of the candidate push products, wherein the visual features are obtained by fusing visual sub-features of each visual data. The visual sub-features are obtained by performing feature extraction based on a target visual instruction on the visual data using a pre-trained investment information push model. The target visual instruction is one of the multiple pre-trained visual instructions corresponding to a target task scenario. The target task scenario is one of the multiple preset task scenarios to which the visual data belongs. There is a unique correspondence between one visual instruction and one task scenario. Then, multimodal feature fusion is performed on the text features and the visual features to obtain a fused feature. Then, market sentiment analysis is performed based on the fused feature. Based on the market sentiment analysis result, a market forecast of the candidate push product is performed to obtain a market forecast result of the candidate push product. Then, based on the market forecast result, a push strategy for pushing the candidate push product to the target object is determined, and investment information push is executed according to the push strategy. Since each visual sub-feature is extracted based on the target visual instructions corresponding to the target task scenario to which the visual data belongs, the investment information push model can pay attention to the detailed information in the visual data of different task scenarios, so that the extracted visual sub-features can be more accurate. In this way, when performing market sentiment analysis based on the fusion features, the investment information push model can make market forecasts based on market sentiment based on text features and visual features, while also paying attention to the impact of detailed information in the visual data of different task scenarios on the analysis of market sentiment, thereby making the market forecast results obtained by the investment information push model more accurate, which is conducive to improving the accuracy of the push strategy, and further conducive to pushing more accurate investment information about candidate push products to the target object.

[0023] An information push method, device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the information reasoning method in the embodiments of the present application is described.

[0024] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0025] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0026] The information push method provided in the embodiment of the present application relates to the field of financial technology. The information push method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the information push method, etc., but is not limited to the above forms.

[0027] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0028] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0029] See also Figure 1 , Figure 1 The flowchart of an information push method provided in an embodiment of the present application is shown. In the embodiment of the present application, the information push method may include steps 110 to 140.

[0030] Step 110: Acquire text features of multiple market text information of candidate push products, and visual features of multiple visual data of candidate push products, wherein the visual features are obtained by fusing visual sub-features of each visual data, and the visual sub-features are obtained by performing feature extraction based on target visual instructions on the visual data using a pre-trained investment information push model. The target visual instruction is one of the multiple pre-trained visual instructions corresponding to the target task scenario, and the target task scenario is one of the multiple preset task scenarios to which the visual data belongs. There is a unique correspondence between a visual instruction and a task scenario. Step 120: Perform multimodal feature fusion on the text features and the visual features to obtain fused features; Step 130: Perform market sentiment analysis based on the fusion features, and perform market forecasts for the candidate push products based on the market sentiment analysis results to obtain market forecast results for the candidate push products. Step 140: Determine a push strategy for pushing candidate push products to target objects based on the market forecast results, and execute investment information push according to the push strategy.

[0031] In one embodiment, market text information refers to textual content commenting on a candidate product after its release. This information may include online news, social media commentary, forum discussions, analyst reports, and the like, though this is not specifically limited here. Furthermore, the release time of multiple pieces of text information may vary.

[0032] In one embodiment, the text feature refers to information that represents the overall characteristics of multiple market text information after integrating multiple market text information. The text feature can be a fusion feature, which can be obtained by fusing the text sub-features of each market text information.

[0033] In one embodiment, in the process of obtaining text features of multiple market text information of candidate push products, multiple market text information of candidate push products can be obtained first, and then, text features of each market text information are extracted separately through the investment information push model to obtain text sub-features of each market text information. Then, it is queried whether multiple text sub-features of candidate push products previously obtained have been stored. If so, the multiple text sub-features of candidate push products previously obtained are called, and the multiple text sub-features of candidate push products previously obtained are fused with the multiple text sub-features currently extracted to obtain text features, and the multiple text sub-features currently extracted are stored.

[0034] In one embodiment, if multiple text sub-features of the candidate pushed product are not stored, multiple currently extracted text sub-features are fused to obtain text features, and the multiple currently extracted text sub-features are stored.

[0035] In one embodiment, obtaining multiple market text information of candidate push products may be performed in response to a push task for a target object, or may be performed regularly at a preset time interval, etc., which is not specifically limited here.

[0036] In one embodiment, the feature fusion of multiple text sub-features can be achieved by feature splicing, feature averaging, feature crossing, etc., which is not specifically limited here.

[0037] In one embodiment, visual data refers to visual information about a candidate product after its launch, including comments about the candidate product. The visual data may include news reports, video reviews, product market charts, financial report images, video presentations, or a combination thereof, without limitation.

[0038] In one embodiment, the visual feature refers to information that is obtained by integrating multiple visual data and is used to represent the overall characteristics of multiple market text information.

[0039] In one embodiment, in the process of obtaining visual features of multiple visual data of candidate push products, multiple visual data of candidate push products can be obtained first, and then, feature extraction of the target visual instructions to which each visual data belongs is performed on each visual data through the investment information push model to obtain visual sub-features of each visual data. Then, it is queried whether multiple visual sub-features of candidate push products previously obtained have been stored. If so, the multiple visual sub-features of candidate push products previously obtained are called, and the multiple visual sub-features of candidate push products previously obtained are feature-fused with the multiple visual sub-features currently extracted to obtain visual features, and the multiple visual sub-features currently extracted are stored.

[0040] In one embodiment, if multiple visual sub-features of the candidate pushed product are not stored, the multiple visual sub-features currently extracted are fused to obtain visual features, and the multiple visual sub-features currently extracted are stored.

[0041] In one embodiment, obtaining multiple visual data of candidate push products may be performed in response to a push task for a target object, or may be performed regularly at a preset time interval, etc., which is not specifically limited here.

[0042] In one embodiment, the feature fusion of multiple visual sub-features may be achieved by feature splicing, feature averaging, feature crossing, etc., which is not specifically limited here.

[0043] In one embodiment, the investment information push model refers to a visual language model (VILA) capable of processing visual and textual modal data, performing sentiment analysis-based market forecasts based on the processed data, and then pushing investment information about candidate financial products to target users based on the market forecast results. The investment information push model may include a visual feature extraction network and a text feature extraction network. The text feature extraction network may be a Large Language Model (LLM) model network, a Bidirectional Encoder Representations from Transformers (BERT) network, or the like, while the visual feature extraction network may be a Vision Transformer (ViT) network, a Residual Block network, or the like, without specific limitations herein.

[0044] In one embodiment, visual instructions refer to textual statements used to instruct the investment information push model to perform targeted feature extraction on visual data using specific feature extraction methods for specific task scenarios. Task scenarios refer to application scenarios with specific tasks, such as Visual Question Answering (VQA), Text Visual Question Answering (TextVQA), and image captioning, though these are not specifically limited.

[0045] In one embodiment, the unique correspondence between a visual instruction and a task scenario refers to a state in which each pre-trained visual instruction uniquely corresponds to one of the multiple task scenarios and does not correspond to any task scenario other than the corresponding task scenario.

[0046] In one embodiment, in the process of performing feature extraction on visual data based on target visual instructions through a pre-trained investment information push model, specifically, scene analysis can be performed on each visual data one by one, and the target task scene to which each visual data belongs can be determined among multiple task scenes. Then, for each visual data, the target visual instructions corresponding to the target task scene of the visual data and the visual data itself are input into the investment information push model, so that the investment information push model performs feature extraction on the visual data according to the target visual instructions to obtain the visual sub-features corresponding to the visual data.

[0047] In one embodiment, in the process of performing scene analysis on each visual data one by one and determining the target task scene to which each visual data belongs in multiple task scenes, it can be performed through a pre-trained scene classification model, or it can be performed through the metadata of the visual data and preset scene classification rules, etc., which is not specifically limited here.

[0048] In one embodiment, multimodal feature fusion refers to the operation of integrating and processing features corresponding to different types of data. The multimodal feature fusion process may include operations such as feature alignment (i.e., converting features from different modalities into a shared feature space) and feature fusion, which are not specifically limited here.

[0049] In one embodiment, in the process of performing multimodal feature fusion on text features and visual features to obtain fused features, the text features and visual features can be first feature aligned to obtain aligned text features and aligned visual features, and then the aligned text features and aligned visual features can be feature fused to obtain fused features.

[0050] In one embodiment, performing market sentiment analysis based on fused features refers to a process of extracting features specific to market sentiment information from the fused features to obtain a market sentiment analysis result, wherein the market sentiment analysis result may be a feature vector that can represent market sentiment information.

[0051] In one embodiment, in the process of performing market sentiment analysis based on the fused features, performing market forecasting for the candidate push product based on the market sentiment analysis results, and obtaining the market forecast results for the candidate push product, the market sentiment analysis can be performed based on the fused features to obtain sentiment features. Then, historical market information for the candidate push product can be obtained, feature extraction can be performed on the historical market information to obtain historical market features, and market forecasting can be performed based on the sentiment features and historical market features to obtain the market forecast results. The market forecast results refer to a feature vector that can represent the future market conditions of the candidate push product.

[0052] In one embodiment, a push strategy refers to a specific method for pushing investment recommendations for a candidate push product to a target object. The push strategy may include generating investment information for the candidate push product based on the candidate push product, pushing the investment information for the candidate push product to the target object, and canceling the push of the investment information for the candidate push product.

[0053] In one embodiment, in the process of determining a push strategy for pushing candidate push products to a target object based on market forecast results, and executing investment information push according to the push strategy, specifically, the financial management attributes of the target object can be first obtained, and feature extraction can be performed on the financial management attributes to obtain the object financial management features of the target object. Then, a first similarity between the object financial management features and the market forecast results is calculated, and then the first similarity is compared with a preset similarity threshold. When the first similarity is greater than or equal to the preset similarity threshold, investment information of the candidate push product is formed based on the candidate push product, and the investment information of the candidate push product is pushed to the target object; when the first similarity is less than the preset similarity threshold, the candidate push product is pushed to the investment information of the candidate push product.

[0054] See also Figure 2 , Figure 2 Shown Figure 1 An embodiment of a training step of the investment information push model disclosed in the disclosure, in one embodiment, the information push method provided in the embodiment of the present application is executed by the investment information push model, and the training step may include steps 210 to 240.

[0055] Step 210: Obtain a first sample set, wherein the first sample set includes a plurality of first sample pairs, each of the first sample pairs including a first training label and a visual instruction sample, each task scene corresponds to a plurality of the plurality of visual instruction samples, and the visual instruction samples of the plurality of first sample pairs in each task scene are different from each other; Step 220: Input a first sample pair into the initial model, and obtain a first sample market forecast result through the initial model; Step 230: performing a loss analysis on the visual command sample based on the first sample market prediction result and the first training label to obtain a first loss value, wherein the first loss value is used to adjust the parameters of the initial model and the visual command sample when the first preset training end condition is not met; Step 240: When the first preset training end condition is met, the initial model after parameter adjustment is determined as the investment information push model, and the visual instructions for multiple task scenarios are determined based on multiple adjusted visual instruction samples.

[0056] In one embodiment, each task scenario corresponds to multiple of the multiple visual instruction samples, meaning that at least one first sample pair among the multiple first sample pairs included in the first sample set corresponds to one of the multiple task scenarios. The first sample set is a training dataset for training the information push capability of the investment information push model for each task scenario.

[0057] In one embodiment, the first sample pair refers to a training data set that enables the initial model to complete an information push training process. The initial model refers to the model of the investment information push model before training is completed, and the model architecture of the initial model is the same as the model architecture of the investment information push model. In addition, in addition to including the first training label and the visual instruction sample, the first sample pair may also include a first text sample feature for representing market text sample information of the pushed product sample, a first visual sample feature for representing feature extraction of visual sample data of the same pushed product sample based on the visual instruction sample, and a first historical market sample feature for representing the historical market conditions of the same pushed product sample, etc., which are not specifically limited here.

[0058] In one embodiment, the first training label refers to the theoretical probability that the initial model outputs the correct push strategy based on the first sample market prediction result and the first object sample feature. The first sample market prediction result here refers to the market prediction result of the pushed product sample obtained by the initial model based on the first text sample feature and the first visual sample feature extracted based on the visual instruction sample.

[0059] In one embodiment, in the process of inputting a first sample pair into an initial model and obtaining a first sample market prediction result through the initial model, the first text sample feature and the first visual sample feature of the pushed product sample can be first subjected to multimodal feature fusion to obtain a first fused sample feature. Then, market sentiment analysis can be performed based on the first fused sample feature to obtain a first emotion sample feature. Then, market prediction can be performed based on the first fused sample feature and the first emotion sample feature to obtain a first sample market prediction result.

[0060] In one embodiment, loss analysis of visual instruction samples refers to the process of determining the difference between the probability of the correct push strategy calculated after the initial model uses the visual instruction sample and the probability before the first training label. Here, the correct push strategy refers to the push strategy that the initial model should output after the initial model is input based on a sample pair. The push strategy determined by the initial model can be the same as the correct push strategy or different from the correct push strategy, which is not specifically limited here.

[0061] In one embodiment, adjusting the parameters of the initial model and the visual instruction sample refers to adjusting the parameters of the initial model and adjusting part or all of the text content in the visual instruction sample based on the first loss value. It should be noted that during the process of adjusting the parameters of the initial model, the multimodal feature fusion capability of the initial model may also be adjusted.

[0062] In one embodiment, during the process of adjusting the parameters of the initial model and the visual instruction sample based on the first loss value, the first loss value and the visual instruction sample can be used as input data and input into a pre-trained model with a text reinforcement learning (RL) mechanism. The model then performs text optimization on the visual instruction sample based on the first loss value to obtain an optimized visual instruction sample. The model with a text reinforcement learning (RL) mechanism can be trained using a Q-learning model, a Deep Q-Networks model, a Policy Gradients (PG) network model, a ChatGLM model, or the like, and the specific examples are not limited here.

[0063] In one embodiment, the first preset training end condition may be that the first loss value is less than the first preset loss threshold, or that the maximum first loss value of each task scenario (the maximum first loss value refers to the maximum value of the first loss values ​​of the first sample pair in a task scenario) is less than the preset first loss threshold, etc., which is not specifically limited here.

[0064] In one embodiment, in the process of determining visual instructions for multiple task scenarios based on multiple adjusted visual instruction samples, the scene features of each task scenario can be obtained first. Then, for each task scenario, the instruction features of multiple adjusted visual instruction samples under the task scenario are obtained, and the multiple instruction features are clustered for the scene features of the task scenario. The instruction feature that is closest to the instruction feature after taking the instruction feature as the center is determined, and the instruction feature is determined as the visual instruction for the task scenario.

[0065] In one embodiment, when the first preset training end condition is not met, the parameters of the initial model and the visual instruction sample are adjusted according to the first loss value, and the next first sample pair is input into the adjusted initial model to continue training for the visual instruction sample (i.e., executing steps 220 to 230).

[0066] Since the first loss value can be used to adjust the parameters of the initial model and the visual instruction samples when the first preset training end condition is not met, each visual instruction sample can better match the initial model's ability to understand visual instructions after adjustment. In this way, when the first preset training end condition is met, the determined investment information push model can more accurately extract visual features from the visual data in different task scenarios based on the visual instructions of the multiple task scenarios determined from the multiple adjusted visual instruction samples, thereby more accurately extracting visual features from the visual data in different task scenarios, which is conducive to improving the accuracy of market forecast results.

[0067] See also Figure 3 , Figure 3 It shows Figure 1 Another training step embodiment of the investment information push model disclosed in , in one embodiment, before obtaining the first sample set, the training step may further include steps 310 to 340.

[0068] Step 310: Acquire a second sample set, wherein the second sample set includes a plurality of second sample pairs, the plurality of second sample pairs correspond to at least one task scene, the second sample pairs belonging to the same task scene correspond to the same visual instruction sample, and the second sample pairs include a second training label; Step 320: Input a second sample pair into the initial model, and obtain a second sample market forecast result through the initial model; Step 330: Perform loss analysis on multimodal feature fusion based on the second sample market prediction result and the second training label to obtain a second loss value, wherein the second loss value is used to adjust the parameters of the initial model when the second preset training end condition is not met; Step 340: When the second preset training end condition is met, cancel inputting the next second sample pair into the initial model and obtain the first sample set.

[0069] In one embodiment, the plurality of second sample pairs in the second sample set corresponding to at least one task scenario refers to either a case where multiple of the plurality of sample pairs can correspond one-to-one with multiple task scenarios, or a case where multiple sample pairs correspond to only one of the multiple task scenarios. The second sample set refers to a training dataset that includes multimodal feature fusion capabilities for training an investment information push model.

[0070] In one embodiment, the second sample pair refers to a training data set that enables the initial model to complete a single information push training process. In addition to the second training label, the second sample pair may also include a second text sample feature representing market text sample information of the pushed product sample, a second visual sample feature corresponding to visual sample data representing the same pushed product sample, and a second historical market price sample feature representing historical market prices for the same pushed product sample, etc., without further limitation herein.

[0071] In one embodiment, the second sample pairs belonging to the same task scene correspond to the same visual instruction sample, which means that the second visual sample features in the second sample pairs belonging to the same task scene are features extracted through the same visual instruction sample.

[0072] In one embodiment, the second training label refers to the theoretical probability that the initial model outputs the correct push strategy based on the second sample market prediction result and the second object sample feature. The second sample market prediction result here refers to the market prediction result of the pushed product sample obtained by the initial model based on the second text sample feature and the second visual sample feature.

[0073] In one embodiment, in the process of inputting a second sample pair into the initial model and obtaining the second sample market prediction result through the initial model, the second text sample features and the second visual sample features of the pushed product sample can be first subjected to multimodal feature fusion to obtain the second fused sample features. Then, market sentiment analysis can be performed based on the second fused sample features to obtain the second emotion sample features. Then, market prediction can be performed based on the second fused sample features and the second emotion sample features to obtain the second sample market prediction result.

[0074] In one embodiment, loss analysis for multimodal feature fusion refers to the process of determining the difference between the probability of the correct push strategy calculated by the initial model and the second training label. In this process, the push strategy determined by the initial model can be the same as the correct push strategy or different from the correct push strategy, which is not specifically limited here.

[0075] In one embodiment, the second preset training end condition may be that the second loss value is less than the second preset loss threshold, or that the number of training times of the initial model reaches the training number threshold, etc., which is not specifically limited here.

[0076] In one embodiment, when the second preset training end condition is not met, the parameters of the initial model are adjusted according to the second loss value, the next second sample pair is input into the adjusted initial model, and training for multimodal feature fusion is continued (i.e., executing steps 320 to 330).

[0077] In one embodiment, when multiple second sample pairs correspond to multiple task scenarios, during the multimodal feature fusion process, multiple second sample pairs from the same task scenario can be input one by one to train the initial model's preliminary feature alignment capabilities. Multiple second sample pairs from multiple task scenarios can then be input to train the initial model's feature alignment capabilities in different task scenarios. Through progressive multimodal fusion training, the model's ability to handle complex multimodal tasks is enhanced, thereby ensuring feature alignment accuracy while improving its versatility and robustness in practical applications.

[0078] By conducting training for multimodal fusion before training for visual instructions, the impact of differences in visual instruction samples on multimodal feature fusion can be reduced, so that when the initial model is trained using the first sample set and the second sample set respectively, it can focus more on the processes of multimodal feature fusion and feature extraction based on visual instructions, thereby reducing the impact of the two on the other end, and making multimodal feature fusion and feature extraction based on visual instructions more accurate.

[0079] See also Figure 4 , Figure 4 Shown Figure 2 In the process of a sub-step embodiment of step 230, in one embodiment, the first sample market prediction result can be obtained by the initial model under the first model parameters based on the first sample pair, and step 230 may include steps 410 to 420.

[0080] Step 410: Based on the first sample market prediction result, calculate the first prediction probability of the push strategy sample determined by the initial model according to the first sample; Step 420: Calculate a first loss value based on the first prediction probability and the first training label.

[0081] In one embodiment, the push strategy sample refers to a push strategy for pushing the push product sample to the target sample after a first sample pair is input into the initial model and the initial model obtains the first sample market prediction result.

[0082] In one embodiment, when calculating the first predicted probability of the push strategy sample determined by the initial model based on the first sample market prediction result, the first sample market prediction result can be normalized to obtain multiple predicted probabilities, and then the maximum value of the multiple predicted probabilities is determined as the first predicted probability. The normalization process can be performed using an activation function such as a sigmoid function or a softmax function, which is not specifically limited here.

[0083] In one embodiment, the first loss value calculated according to the first prediction probability and the first training label can be expressed by the following formula (1).

[0084] (1) in, Can be used to represent the first loss value; Can be used to represent the first sample pair; Can be used to represent the first training label; Can be used to represent the first sample set; It can be used to indicate that after a first sample in the first sample set is input into the initial model, the probability that the output push strategy sample is the correct push strategy is the expected value of the first training label; can be used to represent the first model parameter; It can be used to represent the probability that after a first sample pair in the first sample set is input into the initial model, the push strategy sample output by the initial model under the first model parameters is the correct push strategy.

[0085] By calculating the first prediction probability and obtaining the first loss value based on the first prediction probability and the first training label, the ability of the initial model to extract visual features based on different visual instruction samples can be better expressed by the first loss value, so that the first loss value can more accurately express the loss of visual feature extraction based on visual instructions, which is conducive to more accurately adjusting the first model parameters of the initial model and improving the accuracy of feature extraction of visual data in different task scenarios.

[0086] See also Figure 5 , Figure 5 Shown Figure 3 In the process of a sub-step embodiment of step 330, in one embodiment, the second sample market prediction result is obtained by the initial model under the second model parameters based on the second sample pair, and step 330 may include steps 510 to 520.

[0087] Step 510: Based on the second sample market prediction result, calculate the second prediction probability of the initial model for the push strategy sample determined according to the second sample; Step 520: Calculate a second loss value based on the second sample market prediction result and the second training label.

[0088] In one embodiment, when calculating the second predicted probability of the push strategy sample determined by the initial model based on the second sample market prediction result based on the second sample, the second sample market prediction result can be normalized to obtain multiple predicted probabilities, and then the maximum value among the multiple predicted probabilities is determined as the second predicted probability. The normalization process can be performed using an activation function such as a sigmoid function or a softmax function, which is not specifically limited here.

[0089] In one embodiment, the second loss value calculated according to the second prediction probability and the second training label can be expressed by the following formula (2).

[0090] (2) in, Can be used to represent the second loss value; Can be used to represent the second sample pair; Can be used to represent the second training label; Can be used to represent the second sample set; It can be used to indicate that after a second sample in the second sample set is input into the initial model, the probability that the output push strategy sample is the correct push strategy is the expected value of the second training label; Can be used to represent the second model parameter; It can be used to represent the probability that after a second sample pair in the second sample set is input into the initial model, the push strategy sample output by the initial model under the second model parameters is the correct push strategy.

[0091] By calculating the second prediction probability and obtaining the second loss value based on the second prediction probability and the second training label, the multimodal feature fusion capability of the initial model for different text features and visual features can be better expressed through the second loss value, thereby enabling the second loss value to more accurately express the feature alignment loss and feature fusion loss, which is conducive to more accurate adjustment of the second model parameters of the initial model.

[0092] In one embodiment, text features are obtained by fusing text sub-features of multiple market text information. The text sub-features are obtained by extracting features of target text instructions from the market text information using an investment information push model. The target text instruction is one of multiple pre-trained text instructions corresponding to a target information type, and the target information type is one of multiple preset information types to which the market text information belongs. A unique correspondence exists between a text instruction and an information type. Because each text sub-feature is extracted based on the target text instruction corresponding to the target information type to which the market text information belongs, the investment information push model is able to take into account detailed information in market text information of different information types, thereby making the extracted text sub-features more accurate. In this way, when performing market sentiment analysis based on the fused features, the investment information push model is able to take into account the combined impact of detailed information in market text information of different information types and detailed information in visual data of different task scenarios on market sentiment, thereby making the market forecast results obtained by the investment information push model more accurate, which is conducive to improving the accuracy of the push strategy and, in turn, facilitating more accurate push of candidate push products to the target audience.

[0093] In one embodiment, a text instruction refers to a textual statement used to instruct the investment information push model to extract specific features from market text information using a specific feature extraction method, targeting specific information types. Information types refer to the textual form of market text information, such as questions and answers, guidance text (e.g., notices, financial management articles), etc., and are not specifically limited here.

[0094] In one embodiment, in the process of performing feature extraction on market text information based on target text instructions through a pre-trained investment information push model, the information type of each market text information can be analyzed one by one, and the target information type to which each market text information belongs can be determined among multiple information types. Then, for each market text information, the target text instructions corresponding to the target task scenario of the market text information and the market text information itself are input into the investment information push model, so that the investment information push model performs feature extraction on the market text information according to the target text instructions to obtain the text sub-features corresponding to the market text information.

[0095] In one embodiment, the process of performing information type analysis on each market text information one by one and determining the target information type to which each market text information belongs among multiple information types can be performed through a pre-trained text classification model, or can be performed through the metadata of the market text information and preset text classification rules, etc., which is not specifically limited here.

[0096] In one embodiment, the unique correspondence between a text instruction and an information type refers to a state in which each pre-trained text instruction uniquely corresponds to one of the multiple information types and does not correspond to any information type other than the corresponding information type.

[0097] In one embodiment, the training method of the text instructions may refer to the training method of the visual instructions in steps 210 to 240, which will not be described in detail here.

[0098] See also Figure 6 , Figure 6 Shown Figure 1 Another training step embodiment of the investment information push model disclosed in , in one embodiment, after obtaining the first loss value, the training step may further include steps 610 to 620.

[0099] Step 610: When the first preset training end condition is not met, determine a target first sample pair from the plurality of first sample pairs that belongs to the same task scenario as the current first sample pair and has not been input into the initial model; Step 620: Based on the adjusted visual instruction sample in the current first sample pair, the visual instruction sample and the first visual sample features in the target first sample pair are updated, and the next first sample pair is input into the adjusted initial model.

[0100] In one embodiment, updating the visual instruction samples and the first visual sample features in the target first sample pair based on the adjusted visual instruction samples in the current first sample pair refers to replacing the visual instruction samples in the target first sample pair with the adjusted visual instruction samples in the current first sample pair, and updating the original first visual sample features in the target first sample pair to the first visual sample features obtained by feature extraction of the first visual sample data corresponding to the original first visual sample features based on the initial model and the adjusted visual instruction samples in the current first sample pair.

[0101] The following is a specific example to illustrate steps 710 to 720: assuming that there are 4 first sample pairs, the visual instruction sample in the first first sample pair is "extract the content of this picture", the visual instruction sample in the second first sample pair is "what is the content of this video?", the third first sample pair and the first first sample pair belong to the same task scene, the visual instruction sample in the third first sample pair is "summarize the content of this picture", the fourth first sample pair and the second first sample pair belong to the same task scene, the visual instruction sample in the fourth sample pair is "what is this video about?", assuming that after the first first sample pair is input into the initial model, the first preset training end condition is not met, at this time, the initial model and the visual instruction sample "extract the content of this picture" can be adjusted. At this time, the third first sample pair is determined as the target first sample pair. Assuming that the first first sample pair The visual instruction sample in the sample pair is adjusted to "extract all the content in this picture". At this time, the third first sample pair in the third first sample pair updates "summarize the content of this picture" to "extract all the content in this picture", and then inputs the second first sample pair into the initial model. Assuming that the second first sample pair does not meet the first preset training end condition after inputting the initial model, the initial model and the visual instruction sample "What is the content of this video?" can be adjusted. At this time, the fourth first sample pair is determined as the target first sample pair. Assuming that the visual instruction sample in the second first sample pair is adjusted to "What parts does the content of this video include?", at this time, the third first sample pair in the fourth first sample pair updates "What does this video talk about?" to "What parts does the content of this video include?", and then inputs the third first sample pair into the initial model.

[0102] In one embodiment, when the visual instruction samples in the target first sample pair are updated to the adjusted visual instruction samples in the current first sample pair, in the process of determining the visual instructions for multiple task scenarios based on multiple adjusted visual instruction samples, the visual instruction samples in the target first sample pair under each task scenario can be specifically determined as the visual instructions under each task scenario.

[0103] By updating the visual instruction samples and the first visual sample features in the target first sample pair based on the adjusted visual instruction samples in the current first sample pair when the first preset training end condition is not met, when the target first sample pair is input into the initial model for training, the initial model can be trained based on the first visual features extracted from the iterative visual instruction samples. Since the first visual sample features in the target first sample pair are extracted based on the iterative visual instruction samples, the initial model can perform iterative training on the visual instruction samples for a task scene, so that the initial model's ability to understand the visual instruction samples of a task scene can be iteratively improved. Since the pre-trained visual instructions in a task scene are iteratively formed by visual instruction samples, the visual language model can extract more accurate visual features of the visual data based on a better understanding of the visual instructions when extracting visual features based on the visual instructions of a task scene, thereby making the market prediction results obtained by the visual language model more accurate, which is conducive to improving the accuracy of the push strategy, and further conducive to more accurately pushing candidate push products to the target object.

[0104] See also Figure 7 The embodiment of the present application further provides an information push device that can implement the above method. The information push device 700 may include: Feature acquisition module 710 can be used to obtain text features of multiple market text information of candidate push products and visual features of multiple visual data of candidate push products, wherein the visual features are obtained by fusing visual sub-features of each visual data. The visual sub-features are obtained by performing feature extraction based on target visual instructions on the visual data using a pre-trained investment information push model. The target visual instruction is one of multiple pre-trained visual instructions corresponding to the target task scenario. The target task scenario is one of multiple preset task scenarios to which the visual data belongs. There is a unique correspondence between a visual instruction and a task scenario. The multimodal feature fusion module 720 can be used to perform multimodal feature fusion on text features and visual features to obtain fused features; The market prediction module 730 can be used to perform market sentiment analysis based on the fusion features, and perform market prediction on the candidate push products based on the market sentiment analysis results to obtain the market prediction results of the candidate push products; The push execution module 740 can be used to determine a push strategy for pushing candidate push products to a target object based on the market forecast results, and execute investment information push according to the push strategy.

[0105] The specific implementation of the information push device is basically the same as the specific embodiment of the above-mentioned information push method, and will not be repeated here.

[0106] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned information push method when executing the computer program. The electronic device can be any smart terminal including a tablet computer, an in-vehicle computer, or the like.

[0107] See Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is shown. The electronic device 800 includes: The processor 801 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application. The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called by the processor 801 to execute the information push method of the embodiments of this application. Input / output interface 803, used to implement information input and output; Communication interface 804, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.); Bus 805 , which transmits information between various components of the device (e.g., processor 801 , memory 802 , input / output interface 803 , and communication interface 804 ); The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0108] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned information push method is implemented.

[0109] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0110] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0111] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0112] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0113] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0114] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0115] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0117] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0120] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. An information push method, characterized in that: The following steps are involved: Acquire text features of multiple market text information of candidate push products, and visual features of multiple visual data of the candidate push products, wherein the visual features are obtained by fusing visual sub-features of each of the visual data, and the visual sub-features are obtained by extracting features of the visual data based on target visual instructions through a pre-trained investment information push model, the target visual instruction is one of the target task scenes corresponding to the multiple pre-trained visual instructions, the target task scene is one of the multiple preset task scenes to which the visual data belongs, and there is a unique corresponding relationship between one visual instruction and one task scene; Performing multimodal feature fusion on the text feature and the visual feature to obtain a fused feature; Performing market sentiment analysis according to the fusion features, and performing market forecasting of the candidate pushed products according to the market sentiment analysis results to obtain market forecasting results of the candidate pushed products; According to the market forecast result, a push strategy for pushing the candidate push product to the target object is determined, and the investment information push is executed according to the push strategy.

2. The method according to claim 1, characterized in that The method is executed by the investment information push model; The investment information push model is trained through the following steps: Acquire a first sample set, wherein the first sample set includes a plurality of first sample pairs, the first sample pairs include a first training label and a visual instruction sample, each of the task scenes corresponds to a plurality of the plurality of the visual instruction samples, and the visual instruction samples of the plurality of the first sample pairs under each of the task scenes are different from each other; Inputting one of the first sample pairs into an initial model, and obtaining a first sample market forecast result through the initial model; Performing a loss analysis on the visual instruction sample according to the first sample market prediction result and the first training label to obtain a first loss value, wherein the first loss value is used to adjust the parameters of the initial model and the visual instruction sample when a first preset training end condition is not met; When the first preset training end condition is met, the initial model after parameter adjustment is determined as the investment information push model, and the visual instructions for multiple task scenarios are determined based on multiple adjusted visual instruction samples.

3. The method according to claim 2, characterized in that Before obtaining the first sample set, the investment information push model is further trained through the following steps: Acquire a second sample set, wherein the second sample set includes a plurality of second sample pairs, the plurality of second sample pairs correspond to at least one task scene, the second sample pairs belonging to the same task scene correspond to the same visual instruction sample, and the second sample pairs include a second training label; Inputting one of the second sample pairs into the initial model, and obtaining a second sample market forecast result through the initial model; Performing a loss analysis on multimodal feature fusion according to the second sample market prediction result and the second training label to obtain a second loss value, wherein the second loss value is used to adjust the parameters of the initial model when a second preset training end condition is not met; When the second preset training end condition is met, the input of the next second sample pair into the initial model is canceled, and the first sample set is obtained.

4. The method according to claim 2, characterized in that: The first sample market forecast result is obtained by the initial model under the first model parameters based on the first sample pair; The performing loss analysis on the visual instruction sample according to the first sample market prediction result and the first training label to obtain a first loss value includes: Based on the first sample market prediction result, calculating the first prediction probability of the initial model for the push strategy sample determined according to the first sample; The first loss value is calculated based on the first prediction probability and the first training label.

5. The method according to claim 3, characterized in that: The second sample market forecast result is obtained by the initial model under the second model parameters based on the second sample pair; The performing loss analysis on multimodal feature fusion according to the second sample market prediction result and the second training label to obtain a second loss value includes: Based on the second sample market prediction result, calculating the second prediction probability of the initial model for the push strategy sample determined according to the second sample; The second loss value is calculated according to the second prediction probability and the second training label.

6. The method according to claim 1, characterized in that The text feature is obtained by fusing text sub-features of multiple market text information. The text sub-features are obtained by extracting features of the market text information for target text instructions through the investment information push model. The target text instruction is one of the target information type corresponding to multiple pre-trained text instructions. The target information type is one of the multiple preset information types to which the market text information belongs. There is a unique correspondence between one text instruction and one information type.

7. The method according to claim 2, characterized in that The first sample pair also includes a first visual sample feature; After obtaining the first loss value, the training step of the investment information push model further includes: When the first preset training end condition is not met, determining, from the plurality of first sample pairs, a target first sample pair that belongs to the same task scenario as the current first sample pair and has not been input into the initial model; Based on the adjusted visual instruction sample in the current first sample pair, the visual instruction sample and the first visual sample features in the target first sample pair are updated, and the next first sample pair is input into the adjusted initial model.

8. An information push device, characterized in that: The device comprises: A feature acquisition module, used to acquire text features of multiple market text information of candidate push products, and visual features of multiple visual data of the candidate push products, wherein the visual features are obtained by fusing visual sub-features of each of the visual data, and the visual sub-features are obtained by extracting features of the visual data based on target visual instructions through a pre-trained investment information push model, wherein the target visual instruction is one of the multiple pre-trained visual instructions corresponding to the target task scene, and the target task scene is one of the multiple preset task scenes to which the visual data belongs, and there is a unique corresponding relationship between one visual instruction and one task scene; A multimodal feature fusion module, used for performing multimodal feature fusion on the text feature and the visual feature to obtain a fused feature; A market prediction module, used to perform market sentiment analysis according to the fusion features, and perform market prediction for the candidate push products according to the market sentiment analysis results to obtain market prediction results for the candidate push products; The push execution module is used to determine a push strategy for pushing the candidate push products to the target object according to the market forecast result, and execute investment information push according to the push strategy.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the information push method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the information pushing method according to any one of claims 1 to 7 is implemented.

Images