Message pushing method and system, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a message pushing method and system, electronic equipment, a storage medium and a program product, and relates to the technical field of computers. The message pushing method comprises the steps of determining a state feature of a target user according to a to-be-pushed goods source message list in a goods source message queue of the target user and feature information of the target user, and determining a target goods source message from the to-be-pushed goods source message list; sending the state characteristics of the user to a preset offline reinforcement learning model, and obtaining a target frequency control interval returned by the preset offline reinforcement learning model; and pushing the target goods source message to the target user according to the target frequency control interval. The embodiment of the invention can provide a technical scheme which can reduce invalid pushing and can optimize the long-term experience of the user.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a message push method, system, electronic device, storage medium, and program product. Background Art

[0002] A push notification system is a software architecture that proactively sends messages to end users and is widely used for mobile user engagement. For freight applications, this system must handle high-concurrency scenarios, such as thousands of new cargo notifications per second. Once a qualified driver is matched, the system delivers this information to the target driver in real time through a multi-channel notification system, facilitating the successful delivery of the transport transaction.

[0003] Pushing messages in the traditional proactive way is highly intrusive to users. Too high a push frequency or poor message quality will seriously damage the user experience, causing users to lose interest in messages, or even turn off push notifications or uninstall the app.

[0004] Currently, rule-based frequency control cannot respond to changes in users and messages in real time, easily resulting in one-size-fits-all over-restrictions or ineffective push notifications. Alternatively, a supervised learning-based algorithmic frequency control system predicts click-through rates based on user and message characteristics and pushes only messages with click-through rates exceeding a threshold. While this can reduce ineffective push notifications, it struggles to optimize the long-term user experience. Summary of the Invention

[0005] The present application provides a message push method, system, electronic device, storage medium and program product to provide a technical solution that can reduce invalid push and optimize the long-term user experience.

[0006] In a first aspect, the present application provides a message push method, the method comprising: Determine the state characteristics of the target user according to the list of delivery source messages to be pushed in the delivery source message queue of the target user and the characteristic information of the target user, and determine the target delivery source message from the list of delivery source messages to be pushed; Sending the user's state features to a preset offline reinforcement learning model, and obtaining a target frequency control interval returned by the preset offline reinforcement learning model; According to the target frequency control interval, the target supply message is pushed to the target user.

[0007] In an optional embodiment, the state of the preset offline reinforcement learning model includes: the state of the target user, the historical profile of the target user, and the source characteristics of the supply source message to be pushed; The actions of the preset offline reinforcement learning model include: the next message frequency control interval of the target user; The observation information of the preset offline reinforcement learning model includes: feedback information of the target user obtained by executing the next message frequency control interval of the target user.

[0008] In an optional embodiment, the initial frequency control interval of the preset offline reinforcement learning model is determined according to the source characteristics of the source message to be pushed and the user's matching score, and according to preset business rules; The initial training samples of the preset offline reinforcement learning model are determined based on the obtained system decision logs and target user feedback logs; Among them, the system decision log includes the status information of the target user, the historical portrait information of the target user, the source feature information of the source message to be pushed, and the next message frequency control interval of the target user; the target user feedback log includes the feedback information of the target user.

[0009] In an optional implementation, the preset offline reinforcement learning model adjusts the next message frequency control interval of the target user through a conservative Q learning algorithm.

[0010] In an optional embodiment, before determining the target user's status characteristics based on the list of delivery source messages to be pushed in the target user's delivery source message queue and the target user's characteristic information, and determining the target delivery source message from the list of delivery source messages to be pushed, the method further includes: Determining characteristic information of the target user based on the acquired historical profile information of the target user; wherein the characteristic information of the target user includes the target user's source preference information and the target user's interactive behavior information; According to the matching degree between the acquired characteristic information of the target user and the candidate supply source messages in the plurality of candidate supply source message sets, a list of supply source messages to be pushed in the supply source message queue of the target user is determined.

[0011] In an optional embodiment, the method further includes: Obtaining feedback information from the target user regarding the pushed target supply message; The state characteristics of the target user, the target frequency control interval and the feedback information are used as new training samples of the preset offline reinforcement learning model to update the preset offline reinforcement learning model.

[0012] In an optional implementation, pushing the target supply message to the target user according to the target frequency control interval includes: Calculate the push time of the target supply source message based on the acquired time of the last push of the supply source message and the target frequency control interval; The target supply source message is pushed to the user based on the push time of the target supply source message.

[0013] In an optional implementation, determining the target supply source message from the list of supply source messages to be pushed includes: Determine an initial delivery source message to be pushed or a priority delivery source message according to at least one delivery source message to be pushed in the list of delivery source messages to be pushed, and use the initial delivery source message to be pushed or the priority delivery source message as the target delivery source message; The sending of the user's state feature to a preset offline reinforcement learning model and obtaining a target frequency control interval returned by the preset offline reinforcement learning model includes: Sending the user's state features to a preset offline reinforcement learning model, and obtaining the initial to-be-pushed delivery source message or the target frequency control interval corresponding to the priority push delivery source message returned by the preset offline reinforcement learning model; The pushing the target supply message to the target user according to the target frequency control interval includes: According to the target frequency control interval corresponding to the initial to-be-pushed delivery source message or the priority delivery source message, the initial to-be-pushed delivery source message or the priority delivery source message is preferentially pushed to the user.

[0014] In a second aspect, the present application provides a message push system, the system comprising a scheduler: The scheduler is used to: determine the status characteristics of the target user and determine the target supply message from the list of supply source messages to be pushed in the supply source message queue of the target user and the characteristic information of the target user; Sending the user's state features to a preset offline reinforcement learning model, and obtaining a target frequency control interval returned by the preset offline reinforcement learning model; According to the target frequency control interval, the target supply message is pushed to the target user.

[0015] In an optional embodiment, the system further includes a model service; The preset offline reinforcement learning model is deployed in the model service; The model service is used to: receiving the state characteristics of the user sent by the scheduler, and running the preset offline reinforcement learning model to obtain the target frequency control interval; The target frequency control interval is sent to the scheduler.

[0016] In an optional implementation, the system scheduler is further configured to: Obtaining feedback information from the target user regarding the pushed target supply message; Using the state characteristics of the target user, the target frequency control interval, and the feedback information as new training samples for the preset offline reinforcement learning model to update the preset offline reinforcement learning model; The model service is also used to: The deployed preset offline reinforcement learning model is updated according to the updated preset offline reinforcement learning model.

[0017] In a third aspect, the present application provides an electronic device comprising: a memory and a processor; the memory is configured to store computer program instructions; the processor is configured to run the computer program instructions, so that the electronic device implements the method described in the first aspect.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium comprising computer program instructions. An electronic device executes the computer program instructions so that the electronic device implements the method described in the first aspect.

[0019] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the steps in the method provided in the first aspect.

[0020] In an embodiment of the present application, the message push method includes determining the target user's status characteristics based on a list of pending delivery source messages in the target user's delivery source message queue and the target user's characteristic information, and then determining a target delivery source message from the list of pending delivery source messages. The user's status characteristics are sent to a preset offline reinforcement learning model, and a target frequency control interval returned by the preset offline reinforcement learning model is obtained. Based on the target frequency control interval, the target delivery source message is pushed to the target user.

[0021] Based on this, the embodiments of the present application utilize a pre-configured offline reinforcement learning model to determine an appropriate target frequency control interval based on the target user's state characteristics (including the user's own characteristics and the information reflected in the list of source messages to be pushed). For example, for users during busy periods or users who are not interested in certain categories, the message frequency control interval will be extended, reducing the frequency of message pushes, avoiding the push of messages that the user is not interested in, and reducing user interruption, thereby improving user acceptance of message push and preventing users from closing push notifications or uninstalling the app due to interruptions from messages.

[0022] Furthermore, this embodiment of the present application can identify target supply messages from a list of pending supply messages and push them based on user profile information (such as supply preferences and message click frequency), thus better meeting the user's actual needs. For example, drivers who frequently transport fresh produce will be prioritized for such supply messages. The system will also schedule push notifications based on their busyness and message-receiving habits, thereby improving the user's experience in obtaining effective information.

[0023] Furthermore, by applying the embodiments of the present application to the real-time processing of highly concurrent cargo source messages, it is possible to quickly match eligible target users (drivers) based on the target users' characteristic information, and timely push the target cargo source messages to the target users based on the optimized frequency control interval, which helps users obtain cargo source information more quickly, improves the completion rate of transportation transactions, and thus increases business volume and revenue. Based on this, it is possible to avoid the one-size-fits-all excessive restrictions or invalid push based on rule-based frequency control, as well as the problem that frequency control based on supervised learning algorithms is difficult to take into account long-term experience. This allows message push resources to be more reasonably allocated, focusing resources on pushing messages that are valuable to users and can promote transactions, thereby improving the effectiveness of message push and resource utilization efficiency.

[0024] Finally, the message push method provided by the embodiments of this application, based on user status characteristics and a pre-configured offline reinforcement learning model, can respond in real time to changes in the user and the delivery source information to be pushed, dynamically adjusting the message push strategy. For example, if a driver's transportation route or transportation preferences change, or a new type of delivery source information appears, the system can quickly respond by adjusting the frequency control interval and the pushed messages, ensuring that the message push always matches the target user's actual situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flowchart of a message push method provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a message push system provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] A push notification system is a software architecture that proactively sends messages to end users and is widely used for mobile user engagement. For freight applications, this system must handle high-concurrency scenarios, such as thousands of new cargo notifications per second. Once a qualified driver is matched, the system delivers this information to the target driver in real time through a multi-channel notification system, facilitating the successful delivery of the transport transaction.

[0027] Pushing messages in the traditional proactive way is highly intrusive to users. Too high a push frequency or poor message quality will seriously damage the user experience, causing users to lose interest in messages, or even turn off push notifications or uninstall the app.

[0028] Currently, rule-based frequency control cannot respond to changes in users and messages in real time, easily resulting in one-size-fits-all over-restrictions or ineffective push notifications. Alternatively, a supervised learning-based algorithmic frequency control system predicts click-through rates based on user and message characteristics and pushes only messages with click-through rates exceeding a threshold. While this can reduce ineffective push notifications, it struggles to optimize the long-term user experience.

[0029] Based on this, the technical concept of the embodiment of the present application is to provide a method for forming user status characteristics based on the historical portrait of the target user and combining the user's real-time behavior in the freight scenario. Afterwards, based on the list of source messages to be pushed, the target source messages are matched with the user status characteristics to provide a technical solution that can balance the message reach efficiency (facilitating transactions) and the long-term user experience (reducing interruptions).

[0030] The technical solutions shown in this application are described in detail below through specific embodiments. It should be noted that the following embodiments can exist independently or in combination with each other, and the same or similar contents will not be repeated in different embodiments.

[0031] Reference Figure 1 , an embodiment of the present application provides a method for pushing information, the method comprising: S101, determining the state characteristics of the target user according to the list of delivery source messages to be pushed in the delivery source message queue of the target user and the characteristic information of the target user, and determining the target supply message from the list of delivery source messages to be pushed.

[0032] In an embodiment of the present application, the cargo source message queue of the target user (which can be a driver) first filters out messages that do not match the driver at all from the global pool of newly added cargo sources (candidate cargo source message set) to avoid invalid data occupying resources, and sorts the candidate cargo source information according to rules such as cargo source release time priority and driver matching priority to ensure that higher-quality cargo sources are given priority in the list to be pushed and are pushed first.

[0033] Optionally, the messages to be pushed in the list of messages to be pushed in the target user's cargo source message queue are messages determined in the global pool of newly added cargo sources based on the matching of transportation route, vehicle type / load, cargo type preference, and order acceptance time.

[0034] Among them, the matching rule for transportation routes can be that the cargo's origin is within the driver's order acceptance range and the destination is a route the driver frequently travels. The matching rule for vehicle type / load can be that the vehicle type required by the source is the driver's vehicle type and the cargo weight is less than or equal to the driver's approved load capacity. The matching rule for cargo type preference can be that the source cargo type belongs to the driver's frequently accepted category and does not require special transportation conditions that the driver does not have. The matching rule for order acceptance time can be that the source loading time is within the driver's frequently accepted order period and the driver is currently online and available to accept orders.

[0035] Based on the above description, you can use the list of supply sources to be pushed to filter out completely unmatched supply sources in advance, avoiding wasting computing power when subsequently determining the target supply source and calculating frequency control.

[0036] It should be understood that in the freight scenario, the user in the embodiment of the present application can be understood as a driver, and the characteristic information of the target user may include the target user's identity and basic attribute characteristics, historical behavior and preference characteristics, current business status characteristics, equipment and scene status, etc.

[0037] The target user's identity attributes are used to represent the target user's identity and may include: Basic identity information; driver's name, contact number, registration time (to determine whether the driver is new or old; new drivers may need more basic guidance on cargo sources).

[0038] Transport qualification information; driver's license type, professional qualification certificate type (such as general freight, dangerous goods transport, directly related to whether dangerous goods sources can be pushed), vehicle binding information (vehicle type, vehicle length, vehicle ownership).

[0039] Permanent residence area information; Permanent residence city / county; for example, if you are permanently stationed in City A, priority will be given to pushing cargo within City A and its surrounding 100 kilometers to reduce the driver's idle driving distance.

[0040] Preference for frequently traveled routes; for example, a fixed round trip between City B and City C will prioritize the delivery of round trip cargo on this route to suit the driver's operating habits.

[0041] Among them, historical behaviors and preferences are generated based on the driver's historical interaction data, reflecting what type of cargo the driver is willing to undertake, and are used to optimize push accuracy and increase click-through rate.

[0042] Optionally, historical behaviors and preferences may include: Preference for product type. For example, if 70% of historical orders are for home appliance shipments, priority will be given to similar products that are easy to load and unload and have low loss. If fresh produce shipments are common, priority will be given to time-sensitive products that require refrigerated trucks.

[0043] Preference for cargo volume. For example, if you have historically received more small to medium-sized cargoes of 3-5 tons, you should avoid pushing bulk cargoes of more than 10 tons that require teaming.

[0044] Transportation distance preference: for example, if 90% of historical orders are short-distance orders of 50-200 kilometers, we will reduce the push of long-distance cargo over 500 kilometers to match the driver's transportation radius habits.

[0045] Price sensitivity: For example, if the historical average order price is 6 yuan / km and the click-through rate for cargo priced below 5 yuan / km is low, priority will be given to pushing high-cost-effective cargo priced between 6 and 8 yuan / km.

[0046] Message click preference, for example: if the proportion of clicks on instant supply reaches 80% in the past 30 days, priority will be given to pushing products with strong immediacy, reducing invalid push of reserved supply.

[0047] Reasons for rejection / ignorance, such as multiple rejections of night loading sources, will automatically filter out such sources after recording this preference; Active search behavior, for example: recent frequent searches for cargo from City A to City B, indicates that the driver currently has transportation needs for this route, and real-time cargo for this route will be pushed first.

[0048] Among them, the current business status characteristics reflect whether the driver is currently capable of taking on new cargo sources, and are used to determine whether and when to push cargo source messages, affecting the effectiveness of the push.

[0049] Optionally, current business status characteristics include: current transportation status and recent order pressure.

[0050] The current transport status can include idle, in transit, or accepted. For example, if a driver is currently en route and expected to arrive at the destination in 2 hours, immediate cargo will not be pushed, but cargo waiting to be loaded near the destination will be pushed after 1.5 hours. If an order has been accepted and requires loading the next day, scheduled cargo near the loading point for the next day will be pushed.

[0051] The current transport status can also include empty or full. For example, when the vehicle is empty, priority is given to delivering goods along the way; when the vehicle is fully loaded, only subsequent goods near the unloading point are delivered to avoid interruptions.

[0052] Recent order pressure includes the number of orders received on the day and the remaining time of the current order.

[0053] Among them, the number of orders received on the day can be exemplified as: if the driver has received 3 orders on the day and the historical daily average number of orders is 3, then no new sources of goods will be pushed to avoid the driver rejecting orders due to overload; if only 1 order is received, the push frequency can be appropriately increased.

[0054] An example of the remaining time for the current order is: if the current order is expected to be unloaded within 1 hour, the supply of goods around the unloading location will be pushed 30 minutes in advance to seamlessly connect the transportation task.

[0055] Optionally, the device and scene status includes device status and scene characteristics.

[0056] The device status can be understood as the app's online status and device type. For example, if the app is active in the foreground, push notifications will be prioritized within the app; if it's running in the background, push notifications will be delivered via the push notification bar; if it's offline, the product can be temporarily stored in the push queue and prioritized for push notifications once it's online.

[0057] Scenario characteristics can include time periods and network status. For example, 6-8 AM is peak loading time for drivers, so push notifications are likely to be ignored and can be reduced. Noon-2-2 PM is a rest period, so only high-priority shipments are pushed. Evening-6-22 PM is a low-order period, so push notifications can be reduced appropriately.

[0058] As an explanation of network status: Messages can be pushed to drivers based on their current network status. For example, if a driver is currently on a highway with a weak network signal, prioritize sending deliveries that are "SMS + App Offline Cache" to avoid message loss due to poor network conditions.

[0059] Optionally, the status characteristics of the target user determined based on the list of delivery source messages to be pushed in the target user's delivery source message queue and the characteristic information of the target user can be understood as a dynamic label that reflects the user's current demand priority, behavioral intention, and decision-making tendency, which is comprehensively judged based on the user's long-term fixed characteristics (such as identity), short-term dynamic behavior (such as recent operations, demand changes) and external related information (such as delivery sources to be pushed, time scenarios).

[0060] In other words, the state characteristics of the target user can be understood as the current state of the target user and what immediate needs the target user has. The state characteristics of the target user change dynamically and can be derived after comprehensive consideration of specific scenarios (such as the queue of sources to be pushed).

[0061] Optionally, the status characteristics of the target user may include the urgency of supply demand, business intention status, scenario adaptation status, etc.

[0062] The urgency of supply demand can be understood as determining whether to prioritize highly compatible supplies based on the target user's immediate need for supplies. The business intent state can be understood as the target user's current core business goal (e.g., finding a long-term, stable supply source or a temporary supply source for replenishment orders). The scenario adaptation state can be understood as the degree of match between the user's current scenario and the supply source, for example, finding the next stop on the way.

[0063] Based on the above description, the target user's status characteristics are used to connect the queue of delivery sources to be pushed with the user's actual needs. By dynamically judging the user's status, blind push can be avoided. For example, immediate loading sources can be pushed to users who are planning to ship, so that the delivery sources to be pushed can more accurately match the user's current needs, ultimately improving the conversion rate of source push and reducing invalid information interference.

[0064] In an embodiment of the present application, the target source of goods message is determined from the list of source of goods messages to be pushed. This is a multi-dimensional screening rule established based on the user demand matching degree, the source of goods value priority, and the adaptability of the user's current status. The source of goods message that best suits the target user (driver) is selected from the preliminarily matched sources of goods.

[0065] Optionally, the target source of goods message can be determined from the list of source of goods messages to be pushed based on multiple dimensions such as vehicle type / load matching, transportation qualification matching, and line range matching.

[0066] S102: Send the state feature of the user to a preset offline reinforcement learning model, and obtain a target frequency control interval returned by the preset offline reinforcement learning model.

[0067] It should be understood that pre-setting means that the offline reinforcement learning model is trained in advance through offline data and deployed in a format that can be directly inferred (such as ONNX, TensorRT), rather than real-time training; offline reinforcement learning is the model learning through historical data what state and what frequency control interval can balance user experience and business conversion.

[0068] It's worth noting that the pre-built offline reinforcement learning model cannot directly accept raw feature labels (e.g., if the user is in an empty state and the cargo is daily necessities). The constructed structured state features must first be converted into a numerical format that the model can recognize, ensuring consistency with the input dimensions and data distribution during training. This numerical format is then sent to the deployed offline reinforcement learning model, which then outputs the target frequency control interval.

[0069] S103: Push the target supply message to the target user according to the target frequency control interval.

[0070] It should be understood that before pushing the target supply source message to the target user, the reference time is first determined, that is, the time when the supply source message was pushed last time. Afterwards, the target supply source message is pushed to the target user based on the reference time and the target frequency control interval.

[0071] Based on the above description, the embodiments of the present application utilize a pre-configured offline reinforcement learning model to determine an appropriate target frequency control interval based on the target user's state characteristics (including the user's own characteristic information and the information reflected in the list of source messages to be pushed). For example, for users during busy periods or users who are not interested in certain categories, the message frequency control interval will be extended, reducing the frequency of message pushes, avoiding the push of messages that the user is not interested in, and reducing user interruption, thereby improving user acceptance of message push and preventing users from closing push notifications or uninstalling the app due to interruptions from messages.

[0072] Furthermore, this embodiment of the present application can identify target supply messages from a list of pending supply messages and push them based on user profile information (such as supply preferences and message click frequency), thus better meeting the user's actual needs. For example, drivers who frequently transport fresh produce will be prioritized for such supply messages. The system will also schedule push notifications based on their busyness and message-receiving habits, thereby improving the user's experience in obtaining effective information.

[0073] Furthermore, by applying the embodiments of the present application to the real-time processing of highly concurrent cargo source messages, it is possible to quickly match qualified drivers based on the target user's characteristic information, and timely push the target cargo source message to the user based on the optimized frequency control interval, which helps users obtain cargo source information more quickly, improve the completion rate of transportation transactions, and thus increase business volume and revenue. Based on this, it is possible to avoid the one-size-fits-all excessive restrictions or invalid push based on rule-based frequency control, as well as the problem that frequency control based on supervised learning algorithms is difficult to take into account long-term experience. This allows message push resources to be more reasonably allocated, focusing resources on pushing messages that are valuable to users and can promote transactions, thereby improving the effectiveness of message push and resource utilization efficiency.

[0074] Finally, the message push method provided by the embodiments of this application is based on user status characteristics and a pre-configured offline reinforcement learning model. It can respond to changes in users and messages in real time and dynamically adjust message push strategies. For example, if a driver's transportation route or transportation preferences change, or if a new type of cargo source information appears on the platform, it can quickly respond by adjusting the frequency control interval and pushed messages to ensure that message push is always in line with the actual situation.

[0075] In an optional implementation, the state of the preset offline reinforcement learning model includes: the state of the target user, the historical portrait of the target user, and the source characteristics of the source message to be pushed.

[0076] The state of the preset offline reinforcement learning model is the collection of all information related to the target user (driver) and the decision object (cargo source) that can be obtained before the model makes an action decision. It is used to enable the model to determine the current scenario and what action (frequency control interval) to output.

[0077] Optionally, the target user's status indicates whether the target user is available to receive the goods or whether they urgently need the goods. This status may include the target user's most recent app activity time, the target user's driving status, the target user's real-time order count, the target user's most recent search time, the target user's real-time push and click statistics, and the interval between the target user's last push.

[0078] The target user's historical profile is used to indicate whether the target user can accept certain types of goods and what kind of goods they are willing to accept. This may include information such as user age, historical channel push click statistics, registered vehicle length and vehicle type, etc.

[0079] The source characteristics of the delivery source message to be pushed may include: source quality score, matching score between target user (driver) and source, source scenario, and message style.

[0080] The cargo quality score can be determined based on shipper credibility, price competitiveness, information completeness, and ease of loading and unloading, with higher scores indicating higher quality. High-quality cargo can be prioritized based on the cargo quality score, with shorter intervals, while low-quality cargo may require longer intervals or not be pushed at all.

[0081] The matching score between drivers and cargo sources can be used to assess the compatibility between the source and the user. For sources with high matching scores, notifications are pushed more frequently, prioritizing their delivery. For sources with low matching scores, notifications are pushed more frequently or not at all to avoid nuisance.

[0082] Supply scenarios are business scenarios for supply (e.g., immediate supply, pre-ordered supply, long-term supply). For immediate supply, the push interval should be shortened to avoid expiration. For pre-ordered supply, a longer push interval can be set to avoid rushing pushes. For long-term supply, a moderate push interval is recommended.

[0083] The message style refers to the presentation format of the push message (e.g., plain text / graphics / short video).

[0084] As a specific example, the target user's status can be: the last APP active time is 15 minutes ago, the driving status is empty, the real-time order number is 0, the last search time is 30 minutes ago, the real-time push click statistics are 2 pushes and 1 click in the current hour (click rate 0.5), and the last push interval is 20 minutes ago.

[0085] The user's historical portrait can be: age is 38 years old, channel historical push click statistics are APP pop-up clicks 35%, SMS clicks 12%, and the registered vehicle length and vehicle type are 9.6-meter high-sided trucks.

[0086] The supply information may be: supply quality score is 92 points, driver-goods matching score is 88 points, supply scenario is immediate goods, and message style is pictures and texts with high matching urgent order labels.

[0087] The actions of the preset offline reinforcement learning model include: the next message frequency control interval of the target user.

[0088] It should be understood that the next message frequency control interval of the target user is a specific execution instruction output by the model based on the user status, which is used to determine how long it will take to push the next target supply message to the user.

[0089] After executing this action, the effect of the action can be judged by observing information such as whether the user views, accepts the order, or complains. For example, if a 10-minute interval leads to an order, the action is valid, while a 5-minute interval leads to a complaint, the action is invalid.

[0090] The observation information of the preset offline reinforcement learning model includes: feedback information of the target user obtained by executing the next message frequency control interval of the target user.

[0091] The reward of the preset offline reinforcement learning model includes defining appropriate reward values ​​based on different feedback from target users in combination with business goals.

[0092] In the freight message push scenario, rewards that align with business objectives are defined for the offline reinforcement learning model. This is used to link user feedback on push notifications with the platform's priorities (conversion, experience, and efficiency). This allows the model to determine which behaviors to encourage and which to avoid during learning.

[0093] Specifically, the target user's feedback such as clicks, orders, complaints, etc. can be converted into calculable values ​​to obtain the above-mentioned appropriate reward value.

[0094] It's important to understand that observations provide the model with criteria for determining the quality of actions. Positive feedback (orders accepted, no complaints) indicates that the action is appropriate for the current state, and the model will strengthen the association between that state and the action. Negative feedback (complaints, push notifications disabled) indicates that the action is inappropriate, and the model will weaken that association and adjust the action in the next similar state. As observation samples accumulate, the model can continuously optimize the mapping between states and actions, evolving from usable to precisely adapted.

[0095] Optionally, the initial frequency control interval of the preset offline reinforcement learning model is determined according to the source characteristics of the initial to-be-pushed delivery source message, the matching score between the source of goods corresponding to the initial to-be-pushed delivery source message and the user, and according to preset business rules; The initial training samples of the preset offline reinforcement learning model are determined based on the obtained system decision logs and target user feedback logs; Among them, the system decision log includes the current status information of the target user, the historical portrait information of the target user, the source feature information of the current source message to be pushed, and the next message frequency control interval of the target user; the target user feedback log includes the feedback information of the target user.

[0096] It should be understood that the initial frequency control interval is a transitional decision-making basis for starting the business before the model has been fully trained offline. It is used to directly map the supply value and user matching degree into the frequency control interval through explainable business rules, so as to avoid outputting extreme decisions due to insufficient data in the initial stage of the model going online.

[0097] Among them, the source characteristics of the initial supply source message to be pushed may include the urgency of the supply: the remaining loading time, for example, ≤1 hour is high urgency, 1-4 hours is medium urgency, and >4 hours is low urgency; and the supply quality score: the comprehensive quality of the supply output by the upstream matching engine, for example, ≥90 points is high quality, 80-90 points is medium quality, and <80 points is low quality.

[0098] The matching score between the source of goods corresponding to the initial pending delivery source message and the user can be calculated based on the user's historical profile and the source characteristics corresponding to the initial pending delivery source message. A higher matching score indicates a higher probability that the target user will accept the source of goods, and a shorter frequency control interval is required.

[0099] Optionally, preset business rules are used to directly map supply source characteristics and matching scores to frequency control intervals to avoid subjective judgment.

[0100] As a specific example, the characteristics of the delivery source to be pushed are: the remaining loading time is 30 minutes (high urgency), the source quality score is 92 points (high quality), and the source-user matching score is 88 points (high match, the user has a 9.6-meter high-sided truck). The preset rule may determine that the initial frequency control interval is a 5-minute interval based on high urgency and high matching.

[0101] For example, the system decision log may include: user ID (such as driver Mr. Li's ID 123); user's current status (driving status is empty, and the number of real-time orders is 0); user's historical profile (vehicle type is 9.6-meter high-rail, and the common route is City A-City B); cargo source characteristics (remaining time is 30 minutes, and the quality score is 92 points); the next message frequency control interval is 5 minutes; and the push time.

[0102] The target user feedback log may include associated identifiers: user ID 123, source ID 456, push time 10:00; contact status APP pop-up window delivered; interaction behavior: viewing at 10:02, order acceptance at 10:05; business result: order completed at 14:30; negative feedback: no complaint, and push not closed.

[0103] Optionally, the preset offline reinforcement learning model adjusts the next message frequency control interval of the target user through a conservative Q learning algorithm.

[0104] In an embodiment of the present application, in the freight message push scenario, the offline reinforcement learning model uses a conservative Q-learning (CQL) algorithm to adjust the next message frequency control interval of the target user. Specifically, CQL can be used to prevent the model from making risky decisions (such as high-frequency harassment of users and omission of high-value cargo sources) due to offline data bias (such as sample imbalance and lack of coverage of extreme scenarios), while ensuring that the strategy can stably balance business conversion and user experience.

[0105] It should be understood that, unlike the goal of traditional Q-learning, CQL adds an additional "conservative term" when estimating the Q-value of an action. This actively reduces the Q-value estimate for actions that appear infrequently or even do not appear in offline data, to avoid overoptimism.

[0106] The CQL algorithm is used to strike a balance between leveraging effective strategies from existing offline data and avoiding unverified high-risk actions, ensuring that the actions (frequency control intervals) output by the model are reliable and safe, and will not lead to extreme errors due to data bias.

[0107] It should be understood that in freight scenarios, offline data often has the problem of more short-interval samples of high-matching freight pushes and fewer long-interval samples of low-matching freight pushes. The conservatism of the CQL algorithm can prevent the model from being overly optimistic about small-sample actions such as short-interval low-matching freight pushes, thereby reducing the risk of harassment.

[0108] In an optional embodiment, before determining the target user's status characteristics based on the list of to-be-pushed delivery source messages in the target user's delivery source message queue and the target user's characteristic information and determining the target delivery source message from the list of to-be-pushed delivery source messages, the method further includes: First, based on the acquired historical portrait information of the target user, the characteristic information of the target user is determined; wherein the characteristic information of the target user includes the target user's source preference information and the target user's interactive behavior information.

[0109] In the embodiment of the present application, the supply preference feature is obtained by analyzing the target user's historical browsing and ordering of supply categories, price ranges, time requirements, etc. to obtain the target user's long-term / short-term supply preferences.

[0110] Interaction behavior features are used to count user message click frequency, average response time, complaint / ignore records, etc., to quantify user acceptance of message reach.

[0111] In the freight message push scenario, the target user's historical profile information is used to determine their characteristic information (source preference, interactive behavior). In order to extract the regular characteristics of what sources of goods the user is interested in (source preference) and how the user interacts with the push (interactive behavior) from the long-term accumulated user historical data, it provides a basic basis for subsequently building user status, screening target source messages, and calculating frequency control intervals.

[0112] Second, according to the matching degree between the acquired characteristic information of the target user and the candidate supply source messages in the candidate supply source message set, a list of supply source messages to be pushed in the supply source message queue of the target user is determined.

[0113] In the freight message push scenario, the list to be pushed is determined based on the matching degree between the target user's characteristic information and the candidate cargo source messages in the candidate cargo source message set. This is to screen out highly adaptable cargo sources that the target user is willing to accept and can undertake from the massive candidate cargo sources, and provide cargo source candidates for subsequent dynamic frequency control push.

[0114] Based on this, the present embodiment uses multi-dimensional quantitative matching to avoid one-size-fits-all screening (e.g., focusing solely on freight costs and ignoring routes), ensuring that the list contains only cargo sources that the target user is both able and willing to accept. It can also filter out unmatched cargo sources, reducing subsequent computational effort and adapting to high-concurrency scenarios such as freight delivery with thousands of candidates per second.

[0115] Optionally, the method further includes: obtaining feedback information from the target user regarding the pushed target supply message; The state characteristics of the target user, the target frequency control interval and the feedback information are used as new training samples of the preset offline reinforcement learning model to update the preset offline reinforcement learning model.

[0116] In this embodiment of the present application, the method can also continuously train and optimize a pre-set offline reinforcement learning model by using user status characteristics, target frequency control intervals, and user feedback information on push messages as new training samples. As time passes and data accumulates, the model can continuously learn and adapt to new user behavior patterns, continuously improving the accuracy and effectiveness of message push and frequency control strategies.

[0117] It's important to understand that feedback allows the model to understand the strengths and weaknesses of its strategies, and the creation of new training samples allows the model to remember these lessons. This creates a complete closed loop of decision-making, feedback, and learning. While user behavior may change over time (e.g., from preferring instant delivery to pre-ordered delivery), new samples continuously reflect these changes, ensuring that the model's strategy remains stable. Furthermore, through negative feedback samples (e.g., complaints), the model can quickly identify harmful strategies (e.g., high-frequency push notifications) and proactively avoid them.

[0118] In a possible implementation, pushing the target supply message to the target user according to the target frequency control interval includes: Calculate the push time of the target supply source message based on the acquired time of the last push of the supply source message and the target frequency control interval; The target supply source message is pushed to the user based on the push time of the target supply source message.

[0119] The time of the last pushed supply source message refers to the last successful triggering time of pushing the supply source message to the current target user, which can be retrieved in real time from the push log system.

[0120] The target frequency control interval refers to the decision result output by the offline reinforcement learning model (such as 10 minutes, 15 minutes).

[0121] Based on this, the embodiment of the present application can execute push at the exact time, ensuring that the user is neither disturbed in advance nor delayed to the point of invalidation of the supply.

[0122] Optionally, before pushing the target supply message to the target user according to the target frequency control interval, the method further includes: Determine an initial delivery source message to be pushed, or a priority delivery source message, according to at least one delivery source message to be pushed in the list of delivery source messages to be pushed; The initial to-be-pushed source message or the priority push source message is pushed to the target user.

[0123] Among them, the initial delivery source message to be pushed can be determined based on information such as the driver-cargo matching degree, the value priority of the source, the urgency of the source expiration, and the user's recent preferences.

[0124] It should be understood that determining and pushing the initial pending delivery source messages can quickly awaken the demand of long-inactive or new users with highly matched and high-value sources of goods, avoiding user loss again due to lack of effective messages; through the feedback of the initial push, fill the gaps in user historical portrait lags or insufficient system data, making subsequent pushes based on target frequency control intervals more accurate.

[0125] In one example, determining the target supply source message from the list of supply source messages to be pushed includes: Determine an initial delivery source message to be pushed or a priority delivery source message according to at least one delivery source message to be pushed in the list of delivery source messages to be pushed, and use the initial delivery source message to be pushed or the priority delivery source message as the target delivery source message; The sending of the user's state feature to a preset offline reinforcement learning model and obtaining a target frequency control interval returned by the preset offline reinforcement learning model includes: Sending the user's state features to a preset offline reinforcement learning model, and obtaining the initial to-be-pushed delivery source message or the target frequency control interval corresponding to the priority push delivery source message returned by the preset offline reinforcement learning model; The pushing the target supply message to the target user according to the target frequency control interval includes: According to the target frequency control interval corresponding to the initial to-be-pushed delivery source message or the priority delivery source message, the initial to-be-pushed delivery source message or the priority delivery source message is preferentially pushed to the user.

[0126] In this embodiment of the present application, the initial pending delivery source message is suitable for scenarios where the user's status is uncertain (such as long-term inactivity, newly registered, or a significant status change), and a highly matching source of goods is needed to "activate the user" and obtain real-time feedback. For example, the initial pending delivery source message can be selected from the pending delivery list to select source messages with high matching and high activation potential for the target user.

[0127] In this embodiment of the present application, prioritizing delivery source messages is applicable when the platform needs to quickly convert high-value goods (e.g., high freight rates, high-reputation goods, or platform subsidies), or when goods are extremely time-sensitive (remaining loading time ≤ 1 hour). Exemplarily, prioritizing delivery source messages involves selecting from a list of items to be pushed that are high-value, time-sensitive, and highly compatible with the target user.

[0128] As a specific example, if the supply source message to be pushed previously is a bad supply, the calculated frequency control interval is 10 minutes, but a better supply source message (the supply source message is pushed first) enters the supply source message list to be pushed, and the frequency control interval calculated based on the better supply source message is 5 minutes, then the next push time will be updated from the last push time + 10 minutes to the last push time + 5 minutes, and the better supply source message will be pushed first.

[0129] Based on this, the embodiment of the present application can comprehensively evaluate the sources of goods to be pushed in multiple dimensions, and select the sources of goods that are most attractive to drivers, have the highest business value and are most timely, so as to avoid the sources of goods being snatched away by other platforms due to conventional frequency control waiting, and balance strategy and business timeliness.

[0130] As a specific example, the pre-built offline reinforcement learning model includes the following: Message push can be modeled as a partially observable Markov decision process.

[0131] State: Contains the target user's real-time status (current hour, recent app activity time, driving status, real-time number of orders, recent search time, real-time push click statistics, and the interval between last pushes), the target user's historical profile (user age, channel historical push click statistics, registered vehicle length and vehicle model), and cargo source information (cargo source quality score returned by the upstream matching engine, driver-cargo matching score, cargo source scenario, and message style).

[0132] Action: Message frequency control duration (i.e., how long after the last push interval can this message be pushed) T∈[minimum push interval T_min, maximum push interval T_max].

[0133] Observation: Feedback from target users, including positive feedback (clicks, transactions, fulfillment, retention) and negative feedback (no clicks, push notifications of inferior products, turning off push notifications, churn, etc.).

[0134] Rewards: Define appropriate reward values ​​based on different feedback from target users and business goals.

[0135] Cold start: Define frequency control intervals using business rules (stratify supply sources based on upstream matching scores and linearly map them to frequency control intervals, with the highest-level supply mapped to T_min, the lowest-level supply mapped to T_max, and the remaining levels mapped to intervals in between). Also, increase exploration by a certain percentage. Collect system decision logs and user feedback logs as initial training samples.

[0136] Offline training and evaluation: To ensure user experience and prevent the system from making risky decisions (e.g., excessive frequency control for good products and excessive frequency control for bad products), the CQL (Conservative Q-Learning) algorithm is used. This algorithm penalizes actions outside the dataset, forcing the policy to learn within the scope of the collected samples, effectively controlling risk. The validation set is used to evaluate whether the frequency control for high-reward actions is shortened and whether the frequency control for low-reward actions is increased.

[0137] Online deployment: Export the trained model as an ONNX (Open Neural Network Exchange) file and deploy it to the system's model service. Each time a message is dispatched, the model directly returns the frequency control interval T as input, using the state s. A certain percentage of noise (usually Gaussian noise) is added to the returned T to facilitate continuous exploration and optimization of the model.

[0138] Continuous learning: Samples generated by the latest strategy are collected daily, trained and deployed offline, and the noise exploration ratio is continuously reduced, so that the strategy gradually converges to the optimal one.

[0139] Based on the above description, the method provided in the embodiment of the present application can solve the push frequency control problem in this scenario. Through offline reinforcement learning, it automatically learns the optimal frequency control intervals for different sources of goods for different drivers, reducing the decision-making burden of drivers (target users).

[0140] The dynamic message scheduling architecture based on frequency control intervals of the present invention is more flexible, timely and efficient than a fixed frequency scheduling architecture, and can ensure that messages are pushed to target users in a timely manner at the best time.

[0141] Secondly, refer to Figure 2 , an embodiment of the present application provides a message push system, the system comprising a scheduler 10: The scheduler 10 is used to: determine the status characteristics of the target user and determine the target source message from the list of to-be-pushed source messages in the source message queue of the target user and the characteristic information of the target user; Sending the user's state features to a preset offline reinforcement learning model, and obtaining a target frequency control interval returned by the preset offline reinforcement learning model; According to the target frequency control interval, the target supply message is pushed to the target user.

[0142] Based on the above description, the system provided in the embodiments of the present application utilizes a pre-configured offline reinforcement learning model to determine an appropriate target frequency control interval based on the target user's state characteristics (including the user's own characteristic information and the information reflected in the list of source messages to be pushed). For example, for users during busy periods or users who are not interested in certain categories, the message frequency control interval will be extended, reducing the frequency of message pushes, avoiding the push of messages that the user is not interested in, and reducing user interruption, thereby improving user acceptance of message push and preventing users from closing push notifications or uninstalling the app due to interruptions from messages.

[0143] Furthermore, the system provided by the embodiments of the present application can identify target supply messages from a list of pending supply messages and push them based on user profile information (such as supply preferences and message click frequency), thus better meeting the user's actual needs. For example, the system will prioritize supply messages for drivers who frequently transport fresh produce, and will schedule push notifications at appropriate intervals based on their busy schedule and message-receiving habits, thereby enhancing the user's experience in obtaining effective information.

[0144] Furthermore, the system provided by the embodiment of the present application is applied to the real-time processing of highly concurrent cargo source messages. It can quickly match qualified drivers based on the characteristic information of the target user, and timely push the target cargo source message to the user based on the optimized frequency control interval, which helps users obtain cargo source information more quickly, improve the completion rate of transportation transactions, and thus increase business volume and revenue. Based on this, it is possible to avoid the one-size-fits-all excessive restrictions or invalid push based on rule-based frequency control, as well as the problem that frequency control based on supervised learning algorithms is difficult to take into account long-term experience. Message push resources can be more reasonably allocated, and key resources can be placed on pushing messages that are valuable to users and can promote transactions, thereby improving the effectiveness of message push and resource utilization efficiency.

[0145] Finally, the message push system provided by the embodiments of this application, based on user status characteristics and a pre-configured offline reinforcement learning model, can respond to changes in users and messages in real time, dynamically adjusting message push strategies. For example, if a driver's transportation route or transportation preferences change, or if a new type of cargo source information appears on the platform, the system can quickly respond by adjusting the frequency control interval and pushed messages, ensuring that message pushes always match the actual situation.

[0146] Optionally, the system further includes a model service 20; The preset offline reinforcement learning model is deployed in the model service 20; The model service 20 is used to: receiving the user status characteristics sent by the scheduler 10 and running the preset offline reinforcement learning model to obtain the target frequency control interval; The target frequency control interval is sent to the scheduler 10 .

[0147] Optionally, the scheduler 10 is further configured to: Obtain feedback information from the target user regarding the pushed target supply message.

[0148] The state characteristics of the target user, the target frequency control interval and the feedback information are used as new training samples of the preset offline reinforcement learning model to update the preset offline reinforcement learning model.

[0149] The model service is also used to: The deployed preset offline reinforcement learning model is updated according to the updated preset offline reinforcement learning model.

[0150] Optionally, the scheduler 10 is further used to determine characteristic information of the target user based on the acquired historical portrait information of the target user; wherein the characteristic information of the target user includes the target user's source preference information and the target user's interactive behavior information.

[0151] Determine the list of supply source messages to be pushed in the supply source message queue of the target user according to the matching degree between the obtained characteristic information of the target user and multiple candidate supply source messages.

[0152] Optionally, the scheduler 10 is further configured to: obtain the feedback information of the target user for the pushed target supply source message.

[0153] Use the status characteristics of the target user, the target frequency control interval, and the feedback information as new training samples of the pre-set offline reinforcement learning model.

[0154] Optionally, the scheduler 10 is specifically configured to: calculate the push time of the target supply source message according to the time of the last pushed supply source message obtained and the target frequency control interval.

[0155] Push the target supply source message to the user based on the push time of the target supply source message.

[0156] Optionally, determine the initial supply source message to be pushed according to at least one supply source message to be pushed in the list of supply source messages to be pushed.

[0157] Push the initial supply source message to be pushed to the target user.

[0158] Refer to Figure 2 , for the matching of each target user (driver) and the supply source message to be pushed, the model will return an optimal frequency control interval T. The system is used to push the supply source message to be pushed at the nearest time point that satisfies the frequency control interval. Among them, the scheduling is divided into two types: immediate scheduling and delayed scheduling. Delayed scheduling is a scheduling that will occur in the future and is implemented through a delay queue. When the target user has not started scheduling, or there are higher-priority messages entering the pool, the new message will trigger immediate scheduling.

[0159] Each time of scheduling, the system gives the frequency control interval T, the current time t, and the last message push time t_last. The scheduling result is determined by the following method: If (t - t_last) >= T, that is, the time since the last push has exceeded the frequency control interval T, send directly; at this time, if there is still goods in the supply source pool (the list of supply source messages to be pushed), calculate the frequency control interval T' of the next supply source to be scheduled, and send a delayed scheduling with t_next = t + T'.

[0160] If (t - t_last) < T, that is, the time since the last push has not exceeded the frequency control interval T, send a delayed scheduling with t_next = t_last + T.

[0161] Through this solution, the system can learn and optimize the optimal frequency control interval for matching targeted delivery source messages to different users, maximizing push revenue. Frequency control is reduced for high-activity users and high-quality messages, while frequency control is increased for low-activity users and low-quality delivery source messages. This ensures that delivery source messages are delivered precisely at the earliest possible time that meets the frequency control interval, significantly improving push efficiency and timeliness. This solution has increased push conversion rates by over 50% across multiple push scenarios, while also significantly improving negative user experience indicators for target users.

[0162] In a third aspect, the present application further provides an electronic device, Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 3 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0163] The processor 122 executes the computer-executable instructions stored in the memory, so that the processor 122 performs the scheme in the above-mentioned method embodiment. The processor 122 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0164] The memory 123 is connected to the processor 122 via a system bus and communicates with the processor 122. The memory 123 is used to store computer program instructions.

[0165] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0166] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. System buses can be divided into address buses, data buses, and control buses. For ease of illustration, the diagram uses only a single thick line, but this does not imply a single bus or type of bus. Transceivers enable communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and non-volatile memory.

[0167] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the method in the above embodiment.

[0168] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the method in the above embodiment.

[0169] In a fifth aspect, an embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when at least one processor executes the computer program, it can implement the technical solution of the method in the above embodiment.

[0170] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or module, which can be electrical, mechanical or other forms.

[0171] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these modules may be selected to implement the solution of this embodiment based on actual needs.

[0172] In addition, the functional modules in the various embodiments of the present application may be integrated into a single processing unit, or each module may exist physically separately, or two or more modules may be integrated into a single unit. The above-mentioned modules may be implemented in the form of hardware or hardware plus software functional units.

[0173] The integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The software function modules stored in a storage medium include a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some of the steps of the methods of various embodiments of the present application.

[0174] It should be understood that the processor described above may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0175] The memory may include a high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk.

[0176] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0177] The storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0178] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0179] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A message push method, characterized in that: The method comprises: Determine the state characteristics of the target user according to the list of delivery source messages to be pushed in the delivery source message queue of the target user and the characteristic information of the target user, and determine the target delivery source message from the list of delivery source messages to be pushed; Sending the user's state features to a preset offline reinforcement learning model, and obtaining a target frequency control interval returned by the preset offline reinforcement learning model; According to the target frequency control interval, the target supply message is pushed to the target user.

2. The method according to claim 1, characterized in that The state of the preset offline reinforcement learning model includes: the state of the target user, the historical profile of the target user, and the source characteristics of the supply source message to be pushed; The actions of the preset offline reinforcement learning model include: the next message frequency control interval of the target user; The observation information of the preset offline reinforcement learning model includes: feedback information of the target user obtained by executing the next message frequency control interval of the target user.

3. The method according to claim 2, characterized in that The initial frequency control interval of the preset offline reinforcement learning model is determined according to the source characteristics of the initial to-be-delivered source message, the matching score between the source of goods corresponding to the initial to-be-delivered source message and the user, and the preset business rules; The initial training samples of the preset offline reinforcement learning model are determined based on the obtained system decision logs and target user feedback logs; Among them, the system decision log includes the status information of the target user, the historical portrait information of the target user, the source feature information of the source message to be pushed, and the next message frequency control interval of the target user; the target user feedback log includes the feedback information of the target user.

4. The method according to claim 2, characterized in that The preset offline reinforcement learning model adjusts the next message frequency control interval of the target user through a conservative Q learning algorithm.

5. The method according to any one of claims 1 to 4, characterized in that Before determining the target user's status characteristics based on the list of delivery source messages to be pushed in the target user's delivery source message queue and the target user's characteristic information, and determining the target delivery source message from the list of delivery source messages to be pushed, the method further includes: Determining characteristic information of the target user based on the acquired historical profile information of the target user; wherein the characteristic information of the target user includes the target user's source preference information and the target user's interactive behavior information; According to the obtained matching degree between the characteristic information of the target user and the candidate supply source messages in the candidate supply source message set, a list of supply source messages to be pushed in the supply source message queue of the target user is determined.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Obtaining feedback information from the target user regarding the pushed target supply message; The state characteristics of the target user, the target frequency control interval and the feedback information are used as new training samples of the preset offline reinforcement learning model to update the preset offline reinforcement learning model.

7. The method according to any one of claims 1 to 4, characterized in that Pushing the target supply message to the target user according to the target frequency control interval includes: Calculate the push time of the target supply source message based on the acquired time of the last push of the supply source message and the target frequency control interval; The target supply source message is pushed to the user based on the push time of the target supply source message.

8. The method according to any one of claims 1 to 4, characterized in that Determining the target supply source message from the list of supply source messages to be pushed includes: Determine an initial delivery source message to be pushed or a priority delivery source message according to at least one delivery source message to be pushed in the list of delivery source messages to be pushed, and use the initial delivery source message to be pushed or the priority delivery source message as the target delivery source message; The sending of the user's state feature to a preset offline reinforcement learning model and obtaining a target frequency control interval returned by the preset offline reinforcement learning model includes: Sending the user's state features to a preset offline reinforcement learning model, and obtaining the target frequency control interval corresponding to the initial to-be-delivered message or the target frequency control interval corresponding to the priority delivery message returned by the preset offline reinforcement learning model; The pushing the target supply message to the target user according to the target frequency control interval includes: According to the target frequency control interval corresponding to the initial to-be-pushed delivery source message or the target frequency control interval corresponding to the priority delivery source message, the initial to-be-pushed delivery source message or the priority delivery source message is preferentially pushed to the user.

9. A message push system, characterized in that: The system includes a scheduler: The scheduler is used to: determine the status characteristics of the target user based on the list of delivery source messages to be pushed in the delivery source message queue of the target user and the characteristic information of the target user, and determine the target delivery source message from the list of delivery source messages to be pushed; Sending the user's state features to a preset offline reinforcement learning model, and obtaining a target frequency control interval returned by the preset offline reinforcement learning model; According to the target frequency control interval, the target supply message is pushed to the target user.

10. The system according to claim 9, characterized in that The system also includes a model service; The preset offline reinforcement learning model is deployed in the model service; The model service is used to: receiving the state characteristics of the user sent by the scheduler, and running the preset offline reinforcement learning model to obtain the target frequency control interval; The target frequency control interval is sent to the scheduler.

11. The system according to claim 9, wherein: The system scheduler is also used to: Obtaining feedback information from the target user regarding the pushed target supply message; Using the state characteristics of the target user, the target frequency control interval, and the feedback information as new training samples for the preset offline reinforcement learning model to update the preset offline reinforcement learning model; The model service is also used to: The deployed preset offline reinforcement learning model is updated according to the updated preset offline reinforcement learning model.

12. An electronic device, characterized in that: include: memory and processor; The memory is configured to store computer program instructions; The processor is configured to execute the computer program instructions so that the electronic device implements the method according to any one of claims 1 to 8.

13. A computer-readable storage medium, characterized in that include: computer program instructions; The electronic device runs the computer program instructions, so that the electronic device implements the method according to any one of claims 1 to 8.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when the computer program is executed by a processor.

Images