A multi-modal perception-based organic agricultural product consumption scene dynamic adaptation method and system

Abstract

This invention discloses a dynamic adaptation method for organic agricultural product consumption scenarios based on multimodal perception. The method is executed by an edge server in the exhibition hall and includes: receiving a display collaboration request sent by a mobile terminal; determining a target video based on the source identifier of the streaming media content and product information; determining whether multiple public display terminals are available; and if a first public display terminal is determined to be available, sending the target video to the first public display terminal for playback and sending the terminal identifier of the first public display terminal to the mobile terminal to notify the user of the first public display terminal. This invention can improve the collaborative display efficiency between mobile streaming media content and public display resources in organic agricultural product exhibition halls and enhance the dynamic adaptation effect in agricultural product consumption scenarios.

Description

Technical Field

[0001] This invention relates to the field of multimodal edge computing technology, and in particular to a method and system for dynamic adaptation to organic agricultural product consumption scenarios based on multimodal perception. Background Technology

[0002] With the advancement of rural industrial development, the integration of rural culture and tourism, and the branding of specialty agricultural products, more and more rural areas are showcasing and promoting their local agricultural products through rural tourism, agricultural product exhibition halls, and specialty product experience centers. For agricultural products with regional characteristics and quality differences, such as organic vegetables, organic fruits, organic grains and oils, honey, and tea, relying solely on shelf displays, printed materials, or manual explanations is insufficient to fully demonstrate their origin environment, production process, quality characteristics, certification information, and consumption scenario value. Therefore, utilizing digital display methods to enhance tourists' awareness and willingness to purchase agricultural products has become an important way to promote agricultural products.

[0003] Currently, some agricultural product exhibition halls are beginning to combine offline displays with online real-time streaming promotions. Public display terminals are typically set up within the halls to continuously play promotional videos, origin introduction videos, or videos of product processing. Simultaneously, visitors may also watch real-time streaming content related to agricultural products via mobile devices during their visit. As a result, a multimodal perception foundation, encompassing video, audio, product information, user-focused products, playback status, and network communication status, is gradually forming within the agricultural product display scenario.

[0004] In the context of organic agricultural product consumption, consumers' purchasing decisions are not only related to product name and price, but also to product quality, certification status, origin display, the credibility of streaming media content, and the current focus. However, existing public display terminals in exhibition halls mostly use a fixed loop playback method. Their playback content typically does not consider the streaming media source currently viewed on the visitor's mobile device, the product they are currently interested in, or whether the mobile device is experiencing playback stuttering, loading delays, playback interruptions, or bitrate drops. Therefore, when a visitor is focusing on a particular agricultural product and the mobile device's playback experience deteriorates, the public display terminals in the exhibition hall struggle to promptly capture that user's interest, resulting in a lack of effective coordination between online real-time promotional content and offline display resources.

[0005] Therefore, it is necessary to provide a dynamic adaptation method for organic agricultural product consumption scenarios based on multimodal perception. This method enables the exhibition hall edge server to determine the target video based on the source identifier of the streaming media content and the product information that the user is interested in when the mobile terminal detects that the streaming media content playback status meets the preset conditions. The video is then pushed to the available public display terminal for playback. At the same time, the dynamic adaptation and stable release of public display terminal resources are achieved through playback request sorting, playback time period, and continuation request mechanisms. Summary of the Invention

[0006] This invention utilizes collaboration between a mobile terminal and an exhibition hall edge server to dynamically control the display of product interest information, streaming media source identifiers, and playback status information generated by the user while watching streaming media content on the public display terminal in the exhibition hall. Specifically, when the mobile terminal detects that the streaming media playback status meets preset conditions, it sends a display collaboration request to the exhibition hall edge server. The exhibition hall edge server determines whether the streaming media source is associated with the exhibition hall based on the streaming media source identifier and identifies the corresponding target video from the video library based on the product information the user is currently interested in. When an available public display terminal is available, the exhibition hall edge server sends the target video to that public display terminal for playback and feeds back the terminal identifier of that public display terminal to the mobile terminal to prompt the user to pay attention to the corresponding public display terminal. Through this method, this invention enables the public display terminal in the exhibition hall to promptly take over the agricultural product content that the user is currently interested in when the mobile terminal's playback experience deteriorates or when exhibition hall-side display collaboration is needed, avoiding the problem of mismatch between displayed content and user interests caused by the fixed loop playback of traditional public display terminals. Furthermore, when multiple public display terminals are occupied, this invention quantifies and sorts display collaboration requests based on stuttering metrics, request waiting time, the number of requests for the same product, and the number of times the same mobile terminal has received service, thereby improving the fairness and controllability of public display resource scheduling. Moreover, this invention allocates playback time slots to target videos and releases public display terminals when the time slot expires without receiving a continuation request, achieving stable release of public display terminals without relying on complex indoor positioning or user identification equipment. Therefore, this invention can improve the collaborative display efficiency between mobile streaming media content and public display resources in organic agricultural product exhibition halls and enhance the dynamic adaptation effect in agricultural product consumption scenarios.

[0007] This invention provides a method for dynamically adapting to organic agricultural product consumption scenarios based on multimodal perception. The method is executed by an edge server in the exhibition hall, and is characterized by comprising:

[0008] The system receives a display collaboration request sent by a mobile terminal. The display collaboration request includes at least a streaming media content source identifier, product information that the user is currently watching, and streaming media content playback status. The display collaboration request is sent by the mobile terminal when it detects that the streaming media content playback status meets preset conditions. The streaming media content source identifier, product information, and streaming media content playback status are obtained by the mobile terminal while the user is watching streaming media content.

[0009] The target video is determined based on the source identifier of the streaming media content and product information;

[0010] Determine whether multiple public display terminals are in an available state; and

[0011] If it is determined that the first public display terminal among multiple public display terminals is in an available state, the target video is sent to the first public display terminal for playback, and the terminal identifier of the first public display terminal is sent to the mobile terminal to notify the user of the first public display terminal.

[0012] In a preferred embodiment, determining the target video based on the streaming media content source identifier and product information includes:

[0013] The pre-stored streaming media content source association table is queried based on the streaming media content source identifier to determine whether the streaming media content source is associated with the exhibition hall;

[0014] When the source of the streaming content is determined to be related to the exhibition hall, the target video corresponding to the product information is retrieved from the video library based on the product information.

[0015] In a preferred embodiment, the method further includes:

[0016] If it is determined that multiple public display terminals are occupied, the display collaboration request will be added to the playback request queue, and the multiple display collaboration requests in the playback request queue will be sorted according to the quantized priority.

[0017] Based on the sorting, determine which display collaboration requests will be responded to first;

[0018] The quantification priority is determined by the following factors: stuttering metric, request waiting time, number of requests for the same product within a preset statistical time window, and number of times the same mobile terminal has obtained public display terminal playback services within a preset time window.

[0019] In a preferred embodiment, the method further includes:

[0020] Before the exhibition hall edge server sends the target video to the first public display terminal, a playback time period is allocated to the target video, and the first public display terminal is released when the exit conditions are met. The exit conditions include: the playback time period expires and no continuation request is received from the mobile terminal.

[0021] In a preferred embodiment, the playback time period is determined by the following factors: the remaining playback duration of the target video, the number of display collaboration requests currently waiting to use the public display terminal, the number of display collaboration requests for the same product within a preset statistical time window, and the number of times the same mobile terminal has obtained public display terminal playback services within a preset time window.

[0022] The remaining playback time of the target video is the length of time from the current playback position to the end position of the video.

[0023] Among them, the number of display collaboration requests currently waiting to use the public display terminal is the number of display collaboration requests in the playback request queue that have not yet been allocated public display terminal playback resources when the playback occupation period is determined.

[0024] This invention provides a dynamic adaptation system for organic agricultural product consumption scenarios based on multimodal perception. The system includes an exhibition hall edge server, characterized in that the exhibition hall edge server includes:

[0025] One or more processors;

[0026] Memory coupled to one or more processors, wherein the one or more processors and the memory are configured as follows:

[0027] The system receives a display collaboration request sent by a mobile terminal. The display collaboration request includes at least a streaming media content source identifier, product information that the user is currently watching, and streaming media content playback status. The display collaboration request is sent by the mobile terminal when it detects that the streaming media content playback status meets preset conditions. The streaming media content source identifier, product information, and streaming media content playback status are obtained by the mobile terminal while the user is watching streaming media content.

[0028] The target video is determined based on the source identifier of the streaming media content and product information;

[0029] Determine whether multiple public display terminals are in an available state; and

[0030] If it is determined that the first public display terminal among multiple public display terminals is in an available state, the target video is sent to the first public display terminal for playback, and the terminal identifier of the first public display terminal is sent to the mobile terminal to notify the user of the first public display terminal.

[0031] In a preferred embodiment, determining the target video based on the streaming media content source identifier and product information includes:

[0032] The pre-stored streaming media content source association table is queried based on the streaming media content source identifier to determine whether the streaming media content source is associated with the exhibition hall;

[0033] When the source of the streaming content is determined to be related to the exhibition hall, the target video corresponding to the product information is retrieved from the video library based on the product information.

[0034] In a preferred embodiment, one or more processors and memories are further configured to:

[0035] If it is determined that multiple public display terminals are occupied, the display collaboration request will be added to the playback request queue, and the multiple display collaboration requests in the playback request queue will be sorted according to the quantized priority.

[0036] Based on the sorting, determine which display collaboration requests will be responded to first;

[0037] The quantification priority is determined by the following factors: stuttering metric, request waiting time, number of requests for the same product within a preset statistical time window, and number of times the same mobile terminal has obtained public display terminal playback services within a preset time window.

[0038] In a preferred embodiment, one or more processors and memories are further configured to:

[0039] Before the exhibition hall edge server sends the target video to the first public display terminal, a playback time period is allocated to the target video, and the first public display terminal is released when the exit conditions are met. The exit conditions include: the playback time period expires and no continuation request is received from the mobile terminal.

[0040] In a preferred embodiment, the playback time period is determined by the following factors: the remaining playback duration of the target video, the number of display collaboration requests currently waiting to use the public display terminal, the number of display collaboration requests for the same product within a preset statistical time window, and the number of times the same mobile terminal has obtained public display terminal playback services within a preset time window.

[0041] The remaining playback time of the target video is the length of time from the current playback position to the end position of the video.

[0042] Among them, the number of display collaboration requests currently waiting to use the public display terminal is the number of display collaboration requests in the playback request queue that have not yet been allocated public display terminal playback resources when the playback occupation period is determined.

[0043] Compared with existing technologies, this invention has the following advantages: Firstly, when the playback experience on mobile terminals deteriorates or collaborative display on the exhibition hall side is needed, the public display terminal in the exhibition hall can promptly take over the agricultural product content that the user is currently interested in, avoiding the problem of mismatch between displayed content and user interests caused by the fixed loop playback of traditional public display terminals. Secondly, when multiple public display terminals are occupied, this invention quantifies and sorts display collaboration requests by using stuttering metrics, request waiting time, the number of requests for the same product, and the number of times the same mobile terminal has received service, thereby improving the fairness and controllability of public display resource scheduling. Thirdly, this invention allocates playback time slots to target videos and releases public display terminals when the time slot expires and no continuation request is received, achieving stable release of public display terminals without relying on complex indoor positioning or user identification equipment. Therefore, this invention can improve the collaborative display efficiency between mobile streaming media content and public display resources in organic agricultural product exhibition halls and improve the dynamic adaptation effect in agricultural product consumption scenarios. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of a scenario according to an embodiment of the present invention.

[0045] Figure 2 This is a system logic block diagram of an exhibition hall edge server according to an embodiment of the present invention.

[0046] Figure 3 This is a flowchart of a method according to an embodiment of the present invention.

[0047] Figure 4 This is a schematic diagram of the initial state of an embodiment of the present invention.

[0048] Figure 5 This is a schematic diagram illustrating the display of a target video on a public display terminal, according to an embodiment of the present invention. Detailed Implementation

[0049] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0050] Figure 1 This is a schematic diagram of a scenario according to an embodiment of the present invention. As shown in the figure, in the scenario of the present invention, user 105 may enter a rural agricultural product exhibition hall during a trip. In the exhibition hall, there are multiple public display terminals 115. In traditional systems, the public display terminals 115 typically play pre-recorded product videos in a loop to promote the products. Further... Figure 1As shown, during the visit, user 105 is constantly monitoring their mobile terminal 110, which may be playing live online streaming content (such as live broadcast content). However, most public display terminals in existing exhibition halls use a fixed loop playback method, and the content they play usually does not take into account the streaming media source currently being viewed on the visitor's mobile terminal, the products they are currently interested in, or whether the mobile terminal is experiencing playback stuttering, loading delays, playback interruptions, or bitrate drops.

[0051] Figure 2 This is a system logic block diagram of an exhibition hall edge server according to an embodiment of the present invention. As shown, the exhibition hall edge server may include a request receiving module 210, a target video determination module 215, a display terminal status determination module 220, a video sending module 225, a queue management module 230, a playback time allocation module 235, and a display terminal release module 240. The request receiving module 210 is used to receive a display collaboration request sent by a mobile terminal. The target video determination module 215 is used to determine the target video based on the streaming media content source identifier and product information. In some embodiments, the target video determination module 215 may query a pre-stored streaming media content source association table based on the streaming media content source identifier to determine whether the corresponding streaming media content source is a streaming media content source associated with the exhibition hall; when it is determined that the streaming media content source is a streaming media content source associated with the exhibition hall, the target video determination module 215 may retrieve the target video corresponding to the product information in the video library 245 based on the product information. The display terminal status determination module 220 is used to determine whether multiple public display terminals are in an available state. For example, the display terminal status determination module 220 can obtain the current playback status, occupied status, or idle status of each public display terminal, and determine the first public display terminal that can be used to play the target video from multiple public display terminals. The video sending module 225 is used to send the target video to the first public display terminal for playback when the first public display terminal is in an available state, and to send the terminal identifier of the first public display terminal to the mobile terminal to prompt the user to pay attention to the first public display terminal. The terminal identifier may include the device identifier of the public display terminal, the location identifier within the exhibition hall, or information used to prompt the user to identify the public display terminal. The queue management module 230 is used to add display collaboration requests to the playback request queue when multiple public display terminals are all in an occupied state, and to sort the multiple display collaboration requests in the playback request queue based on quantified priority. The playback occupation time allocation module 235 is used to allocate a playback occupation time for the target video before the exhibition hall edge server sends the target video to the first public display terminal. The display terminal release module 240 is used to release the first public display terminal when the exit conditions are met.

[0052] In some embodiments, the aforementioned request receiving module 210, target video determining module 215, display terminal status determining module 220, video sending module 225, queue management module 230, playback time allocation module 235, and display terminal release module 240 can be implemented in any suitable form. For example, each of the above modules can be implemented as a software program executed by one or more processors, computer-executable instructions stored in a non-transitory computer-readable storage medium, dedicated hardware circuit, programmable logic device, field-programmable gate array, application-specific integrated circuit, firmware, or any combination of the above forms. The above modules can be integrated in the same computing device or distributed in multiple computing devices to work collaboratively. The above division of modules is only for the convenience of describing the functional implementation of the present invention and does not constitute a limitation on the physical structure, deployment location, or implementation method of the modules. Without departing from the concept of the present invention, the function of one module can be implemented by multiple sub-modules, and the function of multiple modules can also be implemented by the same processing unit, program component, or hardware component.

[0053] Figure 3 This is a flowchart of a method according to an embodiment of the present invention. As shown in the figure, the method of the present invention includes the following steps:

[0054] Step 1: Receive a display collaboration request sent by the mobile terminal. The display collaboration request includes at least the streaming media content source identifier, the product information the user is currently viewing, and the streaming media content playback status. The display collaboration request is sent by the mobile terminal when it detects that the streaming media content playback status meets preset conditions. The streaming media content source identifier, product information, and streaming media content playback status are acquired by the mobile terminal while the user is watching the streaming media content. In a specific example, after entering the agricultural product exhibition hall, the user can watch real-time streaming media content related to agricultural products through their mobile terminal. To improve the promotion effect, with the user's authorization, the streaming media service provider can determine whether the user has entered or is approaching the agricultural product exhibition hall based on the approximate location of the user's mobile terminal, such as GPS signal, base station positioning, wireless network access location, or exhibition hall entrance QR code information, and prioritize pushing real-time streaming media content associated with that exhibition hall to the user. It should be noted that this location determination is only used for streaming media content recommendation or entrance guidance and is not used to subsequently determine whether the user has left the public display terminal. Therefore, this invention does not rely on precise indoor positioning to control the release of the public display terminal. A schematic diagram of a user using a mobile terminal to watch streaming media content can be found in [reference needed]. Figure 4 In one example, the collaboration request may further include the mobile device identifier.

[0055] While a user is watching streaming content, the mobile device can obtain the streaming content source identifier, the product information the user is currently viewing, and the streaming content playback status. The streaming content source identifier can be the room ID, channel ID, content source ID, operator account ID, or event session ID of the live streaming room. For example, the room ID of a live streaming room operated by an exhibition hall could be "ROOM-A001". The product information the user is currently viewing can be the specific product being introduced by the presenter in the streaming content, or product information that the user has clicked on, paused, saved, or expanded for details. For example, product information can include product names such as "organic apples," "organic strawberries," "organic grapes," "honey," and "tea," or it can include product IDs, product categories, or product batch information.

[0056] In some embodiments, the streaming media playback status may include a stuttering metric to indicate the smoothness of playback. The stuttering metric can be determined based on the cumulative video buffering duration within a preset detection time window. For example, if the detection time window is 10 seconds, and the mobile terminal detects a cumulative video buffering duration of B seconds within those 10 seconds, then the stuttering metric C can be determined as follows:

[0057] C = 100 × B / 10.

[0058] For example, in a 5G network with good signal strength, the cumulative video buffering time within a 10-second detection window is typically less than 0.2 seconds, in which case the stuttering metric C can be less than 2. When wireless communication is generally acceptable but still viewable, the cumulative video buffering time within a 10-second detection window may be 0.8 to 1.5 seconds, in which case the stuttering metric C can be 8 to 15. When the 4G network signal is weak or there is significant signal obstruction in the exhibition hall, the cumulative video buffering time within a 10-second detection window may reach 3 to 5 seconds, in which case the stuttering metric C can be 30 to 50. The preset conditions can be determined based on the principle of "whether it has significantly affected the user's viewing experience," and should not be frequently triggered due to short-term network fluctuations. In one example, the preset condition could be: stuttering metric C greater than or equal to 20. In another example, the preset condition could be: stuttering metric values ​​greater than or equal to 15 in two consecutive 10-second detection windows. When a mobile terminal detects that the streaming media content playback status meets preset conditions, it can send a display collaboration request to the exhibition hall edge server.

[0059] Step 2: Determine the target video based on the streaming media content source identifier and product information;

[0060] Step 3: Determine whether multiple public display terminals are in an available state. In one example, the available state of a public display terminal can be defined as: the public display terminal is not currently within a playback period triggered by a display collaboration request, and can switch to the target video at the current time or after the current default playback segment ends. In some embodiments, if a public display terminal is looping a regular promotional video, displaying a fixed image, playing interruptible default exhibition hall promotional content, or is in an idle black screen / standby display state, it can be considered to be in an available state. Conversely, if a public display terminal is playing a target video triggered by another display collaboration request, and the corresponding playback period has not yet expired, it can be considered to be in an occupied state.

[0061] Step 4: If it is determined that the first public display terminal among multiple public display terminals is available, the target video is sent to the first public display terminal for playback, and the terminal identifier of the first public display terminal is sent to the mobile terminal to notify the user of the first public display terminal. A diagram illustrating the playback of the target video on the public display terminal can be found here. Figure 5 .

[0062] In a preferred embodiment, determining the target video based on the streaming media content source identifier and product information includes:

[0063] The system queries a pre-stored streaming media content source association table based on the streaming media content source identifier to determine whether the streaming media content source is associated with the exhibition hall. In a specific example, after receiving a display collaboration request, the exhibition hall edge server can determine the target video based on the streaming media content source identifier and product information. In some embodiments, the exhibition hall edge server pre-stores a streaming media content source association table. This association table may include the streaming media content source identifier, exhibition hall identifier, association status, association validity period, and the range of products allowed to be displayed. For example, the streaming media content source association table may include the following records:

[0064] Table 1

[0065]

[0066] For example, when the streaming media source identifier carried in the collaborative display request is "ROOM-A001", the showroom edge server queries the aforementioned association table to determine that the streaming media source is associated with the current showroom HALL-01, and further retrieves the target video based on product information. If the streaming media source identifier is "ROOM-X999", the showroom edge server can determine that the streaming media source is not associated with the current showroom, and therefore will not perform collaborative playback on the public display terminal, or will return a prompt to the mobile terminal indicating that collaborative display is not possible.

[0067] When the source of the streaming content is determined to be a source associated with the exhibition hall, the target video corresponding to the product information is retrieved from the video library based on the product information. In some embodiments, the video library may be pre-established by the exhibition hall. The exhibition hall may pre-record and edit product introduction video clips for various organic agricultural products, such as videos of the production environment, harvesting process, product appearance, processing and packaging, quality explanation, certification explanation, or consumption scenarios. Each video clip may be associated with one or more product information items. For example, the video library may store fields such as product ID, product name, product category, video file address, video duration, applicable exhibition hall, and review status. For organic agricultural products, in some embodiments, only video clips whose certification status, display content, or product information meets the exhibition hall's preset display rules are set to a searchable state.

[0068] For example, if the product information in the collaboration request is "organic strawberries," the exhibition hall edge server can first accurately retrieve the video based on the product ID. If the request does not include a product ID, it can retrieve the corresponding video based on the product name "organic strawberries." If no match is found, it can retrieve the category introduction video based on the product category "organic fruit." In this way, the exhibition hall edge server can determine the target video corresponding to the product that the user is currently interested in.

[0069] In a preferred embodiment, the method further includes:

[0070] If it is determined that multiple public display terminals are occupied, the display collaboration request will be added to the playback request queue, and the multiple display collaboration requests in the playback request queue will be sorted according to the quantized priority.

[0071] Based on the sorting, determine which display collaboration requests will be responded to first;

[0072] The quantification priority is determined by the following factors: stuttering metric, request waiting time, the number of requests for the same product within a preset statistical time window, and the number of times the same mobile terminal has obtained public display terminal playback services within a preset time window. In a specific example, the request waiting time can be defined as the difference between the current server time and the time when the display collaboration request enters the playback request queue. For example, if request A enters the queue at 10:00:00 and the current server time is 10:00:45, then the request waiting time for request A is 45 seconds.

[0073] The preset statistical time window can be determined based on the exhibition hall's visitor density, average video playback duration, and the rate of change in product attention. For example, in a typical exhibition hall, the preset statistical time window can be 60 seconds; in scenarios with high visitor traffic or rapid changes in product attention, it can be set to 30 seconds; and in scenarios with low visitor traffic, it can be set to 120 seconds. In one example, "the number of requests for the same product within the preset statistical time window" could be the number of display collaboration requests carrying the same product ID or the same product name within the 60 seconds prior to the current time.

[0074] A preset time window can be used to limit the repeated access of the same mobile terminal to public display terminal playback services. This time window can be determined based on the average visitor time in the exhibition hall. For example, in an exhibition hall where the average visitor time is 20 minutes, the preset time window can be 10 minutes; in an exhibition hall with a faster visitor flow, it can be set to 5 minutes; and in an experience hall with a longer visit time, it can be set to 15 minutes or 30 minutes. In one example, "the number of times the same mobile terminal has accessed public display terminal playback services within the preset time window" could be the total number of times the same mobile terminal has accessed initial playback time slots and continued playback time slots within the past 10 minutes.

[0075] In order to enable factors with different dimensions to be calculated in the same formula, the exhibition hall edge server can first normalize each factor so that each factor is converted into a numerical range of 0 to 100.

[0076] In one example, the normalized stuttering metric S can be determined as follows:

[0077] S = min(100, 100 × C / Cref)

[0078] Where C is the lag metric detected by the mobile terminal, and Cref is the preset lag reference upper limit. For example, Cref can be 50. When C is 25, S is 50; when C is greater than or equal to 50, S is truncated to 100.

[0079] The normalized request wait time W can be determined as follows:

[0080] W = min(100, 100 × L / Lref)

[0081] Where L is the waiting time after the display collaboration request enters the playback request queue, and Lref is the preset upper limit of the waiting time. For example, Lref can be 120 seconds. When L is 60 seconds, W is 50; when L is greater than or equal to 120 seconds, W is truncated to 100.

[0082] The normalized number of requests N for the same product can be determined as follows:

[0083] N = min(100, 100 × K / Kref)

[0084] Where K is the number of display collaboration requests for the same product within the preset statistical time window, and Kref is the preset upper limit for the number of product requests. For example, Kref can be 10. When K is 5, N is 50; when K is greater than or equal to 10, N is truncated to 100.

[0085] The normalized number of times R has been served by the same mobile terminal can be determined as follows:

[0086] R = min(100, 100 × E / Eref)

[0087] Where E represents the number of times the same mobile terminal has obtained public display terminal playback services within a preset time window, and Eref represents the preset upper limit for the number of services. For example, Eref can be 3. When E is 1, R is approximately 33; when E is greater than or equal to 3, R is truncated to 100.

[0088] In a specific sorting example, multiple public display terminals are all occupied, and there are 5 display collaboration requests in the playback request queue. The exhibition hall edge server can normalize the stuttering metric, request waiting time, number of requests for the same product, and number of times the same mobile terminal has received service, and calculate the quantified priority P according to the following formula:

[0089] P = 0.40S + 0.25W + 0.25N - 0.10R.

[0090] Where S is the normalized stuttering metric, W is the normalized request wait time, N is the normalized number of requests for the same product, and R is the normalized number of times the same mobile terminal has received service. Example data is as follows:

[0091] Table 2

[0092]

[0093] In this example, request D has the highest priority, so when any public display terminal is released, the exhibition hall edge server can respond to request D first. Next, it can respond to requests C, B, A, and E in that order. This prioritization method allows the system to balance factors such as playback stuttering, fairness during waiting times, shared user interest in the same product, and prevention of the same mobile terminal repeatedly occupying public display resources.

[0094] In a preferred embodiment, the method further includes:

[0095] Before the exhibition hall edge server sends the target video to the first public display terminal, a playback time slot is allocated for the target video. The first public display terminal is released when exit conditions are met, including the expiration of the playback time slot and the absence of a continuation request from the mobile terminal. The purpose of setting a playback time slot is to: prevent the public display terminal from being occupied by a single display collaboration request for an extended period; promptly release the public display terminal when the user no longer pays attention or interacts via the mobile terminal; and achieve stable resource release control through continuation requests from the mobile terminal without relying on GPS, indoor positioning, camera recognition, or other complex sensing devices. In other words, if the user is indeed paying attention to the target video playing on the public display terminal, they can trigger a continuation request via the mobile terminal; if the user does not pay attention, leaves, or no longer operates the mobile terminal, the exhibition hall edge server releases the first public display terminal after the playback time slot expires because no continuation request has been received.

[0096] In a preferred embodiment, the playback time period is determined by the following factors: the remaining playback duration of the target video, the number of display collaboration requests currently waiting to use the public display terminal, the number of display collaboration requests for the same product within a preset statistical time window, and the number of times the same mobile terminal has obtained public display terminal playback services within a preset time window.

[0097] The remaining playback time of the target video is the length of time from the current playback position to the end position of the video.

[0098] The number of display collaboration requests currently waiting to use the public display terminal is the number of display collaboration requests in the playback request queue that have not yet been allocated playback resources on the public display terminal when the playback occupation period is determined. In some embodiments, the playback occupation period T can be determined by the remaining playback duration D of the target video, the normalized number of currently waiting requests Qn, the normalized number of requests for the same product Mn, and the normalized number of times the same mobile terminal has been served Hn. Here, D is the time length between the current playback position and the end position of the target video, and Qn, Mn, and Hn are all normalized values ​​in the range of 0 to 100. It is understood that when the target video is first played on the public display terminal, the current playback position is the starting position of the target video; when the target video continues to play due to a continuation request, the current playback position is the position that the public display terminal has already played to.

[0099] The normalized number of currently waiting requests, Qn, can be determined as follows:

[0100] Qn = min(100, 100 × Q / Qref)

[0101] Where Q represents the number of display collaboration requests currently waiting to use the public display terminal, and Qref is the preset upper limit for the number of waiting requests. For example, Qref can be 10. When Q is 5, Qn is 50; when Q is greater than or equal to 10, Qn is truncated to 100.

[0102] The normalized number of requests Mn for the same product can be determined as follows:

[0103] Mn = min(100, 100 × K / Kref)

[0104] Where K is the number of display collaboration requests for the same product within the preset statistical time window, and Kref is the preset upper limit for the number of product requests. For example, Kref can be 10.

[0105] The normalized number of times Hn has been served by the same mobile terminal can be determined as follows:

[0106] Hn = min(100, 100 × E / Eref)

[0107] Where E represents the number of times the same mobile terminal has obtained public display terminal playback services within a preset time window, and Eref represents the preset upper limit for the number of services. For example, Eref can be 3.

[0108] In some embodiments, the playback time period T can be determined according to the following formula:

[0109] T = min(D, Tmax, max(Tmin, T0 - αQn + βMn - γHn))

[0110] Where Tmin is the minimum single playback time period, Tmax is the maximum single playback time period, T0 is the baseline playback duration, and α, β, and γ are preset coefficients.

[0111] The purpose of Tmin is to prevent a single playback session from being too short. If the playback session is too short, the public display terminal may be released immediately after switching to the target video, making it difficult for the user to recognize the correspondence between the target video and the product currently being viewed on their mobile device. Therefore, Tmin can be determined based on the video switching time of the public display terminal, the time required for the user to recognize the displayed content, and the time required for the mobile device to prompt the user to pay attention to the public display terminal. For example, if the public display terminal takes about 1 second to switch videos, the user takes about 3 seconds to recognize the product image, and the mobile device prompts the user and takes about 2 seconds, then Tmin can be set to 5 to 8 seconds. In one example, Tmin can be 5 seconds.

[0112] The purpose of Tmax is to prevent a single display collaboration request from occupying a public display terminal for an extended period. Since public display terminals are shared resources in the exhibition hall, if a single display collaboration request can occupy them for too long, it may affect the response to subsequent display collaboration requests. Therefore, Tmax can be determined based on the number of public display terminals, the average number of queued requests during peak hours, the maximum expected waiting time in the exhibition hall, and the typical length of the target video segment. For example, if the exhibition hall wants a faster turnaround time for queued requests, Tmax can be set to 20 to 30 seconds; if there are many public display terminals or low foot traffic, Tmax can be appropriately increased. In one example, Tmax can be 30 seconds.

[0113] It's important to note that Tmax limits the time period occupied by a single playback, not the cumulative playback duration of the target video. If the target video has not finished playing, and the mobile terminal sends a continuation request before the playback time period expires, the exhibition hall edge server can recalculate the next playback time period, allowing the target video to continue playing.

[0114] In one example, Tmin is 5 seconds, Tmax is 30 seconds, T0 is 15 seconds, α is 0.08, β is 0.15, and γ is 0.12.

[0115] In the first specific example, the remaining playback time D of the target video is 25 seconds, the current number of waiting requests Q is 0, and the preset upper limit for the number of waiting requests Qref is 10, therefore Qn is 0; the number of display collaboration requests for the same product within the preset statistical time window K is 2, Kref is 10, therefore Mn is 20; the number of times the same mobile terminal has received services E is 0, Eref is 3, therefore Hn is 0. Therefore:

[0116] T = min(25, 30, max(5, 15 - 0.08×0 + 0.15×20 - 0.12×0))

[0117] T = min(25, 30, 18)

[0118] T = 18 seconds

[0119] In this scenario, the public display terminal resources are relatively idle, and the mobile terminal has not yet occupied the playback service, so an 18-second playback period can be allocated. If the mobile terminal sends a continuation request before the expiration of this period, the next playback period can be allocated to the target video; if no continuation request is received, the public display terminal is released.

[0120] In the second specific example, the remaining playback time D of the target video is 40 seconds, the current number of waiting requests Q is 8, Qref is 10, therefore Qn is 80; the number of display collaboration requests for the same product within the preset statistical time window K is 2, Kref is 10, therefore Mn is 20; the number of times the same mobile terminal has received services E is 0, Eref is 3, therefore Hn is 0. Therefore:

[0121] T = min(40, 30, max(5, 15 - 0.08×80 + 0.15×20 - 0.12×0))

[0122] T = min(40, 30, 11.6)

[0123] T = 11.6 seconds

[0124] In this scenario, although the target video still has a considerable remaining duration, due to the large number of requests waiting in the queue, the system shortens the playback time for each current request to improve the turnaround efficiency of the public display terminal. This design does not mean that the 40-second target video cannot be played completely; if the mobile terminal continues to send continuation requests, the server can continue to allocate subsequent playback time slots.

[0125] In the third specific example, the remaining playback time D of the target video is 50 seconds, the current number of waiting requests Q is 2, Qref is 10, therefore Qn is 20; the number of display collaboration requests for the same product within the preset statistical time window K is 7, Kref is 10, therefore Mn is 70; the number of times the same mobile terminal has received services E is 0, Eref is 3, therefore Hn is 0. Therefore:

[0126] T = min(50, 30, max(5, 15 - 0.08×20 + 0.15×70 - 0.12×0))

[0127] T = min(50, 30, 23.9)

[0128] T = 23.9 seconds

[0129] In this scenario, although there are waiting requests, the fact that the same product is requested by multiple mobile terminals within the statistical time window indicates that the product has high group attention. Therefore, the system can appropriately extend the playback time period.

[0130] In the fourth specific example, the remaining playback time D of the target video is 35 seconds, the current number of waiting requests Q is 3, Qref is 10, therefore Qn is 30; the number of display collaboration requests for the same product within the preset statistical time window K is 2, Kref is 10, therefore Mn is 20; the number of times the same mobile terminal has received services E is 2, Eref is 3, therefore Hn is approximately 67. Therefore:

[0131] T = min(35, 30, max(5, 15 - 0.08×30 + 0.15×20 - 0.12×67))

[0132] T = min(35, 30, max(5, 7.56))

[0133] T = 7.56 seconds

[0134] In this scenario, since the same mobile terminal has already received playback services multiple times within a preset time window, the system shortens the subsequent playback time to avoid a single mobile terminal continuously occupying public display terminal resources.

[0135] Before the playback period expires, the mobile terminal can display a continuation prompt to the user. For example, 3 seconds before the playback period expires, the mobile terminal can prompt the user whether to continue watching the product video on the corresponding public display terminal. If the user confirms, the mobile terminal can send a continuation request to the exhibition hall edge server. After receiving the continuation request, the exhibition hall edge server can re-determine a new playback period based on the remaining playback time of the target video, the number of currently waiting requests, the number of requests for the same product, and the number of times the same mobile terminal has been served. If the playback period expires and the exhibition hall edge server does not receive a continuation request from the mobile terminal, it releases the first public display terminal, allowing it to resume playing the default promotional content, or plays the target video corresponding to the next display collaboration request in the queue. Those skilled in the art will understand that the continuation request may include at least one of the following: mobile terminal identifier, public display terminal identifier, target video identifier, and playback session identifier, so that the exhibition hall edge server can determine the playback period corresponding to the continuation request.

[0136] This invention provides a dynamic adaptation system for organic agricultural product consumption scenarios based on multimodal perception. The system includes an exhibition hall edge server, characterized in that the exhibition hall edge server includes:

[0137] One or more processors;

[0138] Memory coupled to one or more processors, wherein the one or more processors and the memory are configured as follows:

[0139] The system receives a display collaboration request sent by a mobile terminal. The display collaboration request includes at least a streaming media content source identifier, product information that the user is currently watching, and streaming media content playback status. The display collaboration request is sent by the mobile terminal when it detects that the streaming media content playback status meets preset conditions. The streaming media content source identifier, product information, and streaming media content playback status are obtained by the mobile terminal while the user is watching streaming media content.

[0140] The target video is determined based on the source identifier of the streaming media content and product information;

[0141] Determine whether multiple public display terminals are in an available state; and

[0142] If it is determined that the first public display terminal among multiple public display terminals is in an available state, the target video is sent to the first public display terminal for playback, and the terminal identifier of the first public display terminal is sent to the mobile terminal to notify the user of the first public display terminal.

[0143] In a preferred embodiment, determining the target video based on the streaming media content source identifier and product information includes:

[0144] The pre-stored streaming media content source association table is queried based on the streaming media content source identifier to determine whether the streaming media content source is associated with the exhibition hall;

[0145] When the source of the streaming content is determined to be related to the exhibition hall, the target video corresponding to the product information is retrieved from the video library based on the product information.

[0146] In a preferred embodiment, one or more processors and memories are further configured to:

[0147] If it is determined that multiple public display terminals are occupied, the display collaboration request will be added to the playback request queue, and the multiple display collaboration requests in the playback request queue will be sorted according to the quantized priority.

[0148] Based on the sorting, determine which display collaboration requests will be responded to first;

[0149] The quantification priority is determined by the following factors: stuttering metric, request waiting time, number of requests for the same product within a preset statistical time window, and number of times the same mobile terminal has obtained public display terminal playback services within a preset time window.

[0150] In a preferred embodiment, one or more processors and memories are further configured to:

[0151] Before the exhibition hall edge server sends the target video to the first public display terminal, a playback time period is allocated to the target video, and the first public display terminal is released when the exit conditions are met. The exit conditions include: the playback time period expires and no continuation request is received from the mobile terminal.

[0152] In a preferred embodiment, the playback time period is determined by the following factors: the remaining playback duration of the target video, the number of display collaboration requests currently waiting to use the public display terminal, the number of display collaboration requests for the same product within a preset statistical time window, and the number of times the same mobile terminal has obtained public display terminal playback services within a preset time window.

[0153] The remaining playback time of the target video is the length of time from the current playback position to the end position of the video.

[0154] Among them, the number of display collaboration requests currently waiting to use the public display terminal is the number of display collaboration requests in the playback request queue that have not yet been allocated public display terminal playback resources when the playback occupation period is determined.

[0155] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to specific embodiments, those skilled in the art should understand that various modifications, substitutions, combinations, splits, or equivalent modifications can be made to the technical features in the above embodiments without departing from the spirit and substance of the present invention, and such modifications, substitutions, combinations, splits, or equivalent modifications should all fall within the scope of protection of the present invention. It should be understood that the method steps, modules, units, or functional components described herein can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the relevant functions can be stored as one or more instructions or code in a non-transitory computer-readable storage medium and executed by one or more processors. The computer-readable storage medium may include, but is not limited to, read-only memory, random access memory, flash memory, disk, optical disk, solid-state memory, or other media capable of storing computer-executable instructions. It should also be understood that the terms "first" and "second" used herein are only used to distinguish different objects and do not indicate a specific order, priority, or quantity limitation. Unless the context clearly states otherwise, the terms "comprising," "including," or any variations thereof used herein are intended to cover a non-exclusive inclusion, such that a process, method, system, apparatus, or device that comprises a list of elements may include not only the elements expressly listed but also other elements not expressly listed. The technical features of the various embodiments herein can be combined in any reasonable way according to actual needs, as long as such combinations do not contradict each other. Therefore, the scope of protection of this invention should not be limited to the specific embodiments disclosed in the specification, but should be determined by the scope defined in the claims.

Claims

1. A method for dynamic adaptation to organic agricultural product consumption scenarios based on multimodal perception, wherein the method is executed by an edge server in an exhibition hall, characterized in that, The method includes: The system receives a display collaboration request sent by a mobile terminal, wherein the display collaboration request includes at least a streaming media content source identifier, product information that the user is currently watching, and streaming media content playback status, wherein the display collaboration request is sent by the mobile terminal when it detects that the streaming media content playback status meets preset conditions, and wherein the streaming media content source identifier, the product information, and the streaming media content playback status are obtained by the mobile terminal during the user's viewing of streaming media content; The target video is determined based on the streaming media content source identifier and the product information; Determine whether multiple public display terminals are in an available state; and If it is determined that the first public display terminal among multiple public display terminals is in an available state, the target video is sent to the first public display terminal for playback, and the terminal identifier of the first public display terminal is sent to the mobile terminal to prompt the user about the first public display terminal.

2. The method according to claim 1, wherein, Based on the streaming media content source identifier and the product information, the target video is determined to include: Based on the streaming media content source identifier, a pre-stored streaming media content source association table is queried to determine whether the streaming media content source is a streaming media content source associated with the exhibition hall; When it is determined that the source of the streaming media content is a streaming media content source associated with the exhibition hall, the target video corresponding to the product information is retrieved from the video library based on the product information.

3. The method according to claim 1, wherein, The method further includes: If it is determined that multiple public display terminals are occupied, the display collaboration request is added to the playback request queue, and the multiple display collaboration requests in the playback request queue are sorted according to the quantized priority. Based on the sorting, the display collaboration requests that will be responded to first will be determined; The quantification priority is determined by the following factors: stuttering metric, request waiting time, number of requests for the same product within a preset statistical time window, and number of times the same mobile terminal has obtained public display terminal playback services within a preset time window.

4. The method according to claim 1, wherein, The method further includes: Before the exhibition hall edge server sends the target video to the first public display terminal, a playback time slot is allocated for the target video, and the first public display terminal is released when the exit conditions are met. The exit conditions include: the playback time slot expires and no continuation request is received from the mobile terminal.

5. The method according to claim 4, wherein, The playback time period is determined by the following factors: the remaining playback duration of the target video, the number of display collaboration requests currently waiting to use the public display terminal, the number of display collaboration requests for the same product within a preset statistical time window, and the number of times the same mobile terminal has obtained public display terminal playback services within a preset time window. Wherein, the remaining playback time of the target video is the length of time from the current playback position to the end position of the video; The number of display collaboration requests currently waiting to use the public display terminal is the number of display collaboration requests in the playback request queue that have not yet been allocated public display terminal playback resources when the playback occupation period is determined.

6. A dynamic adaptation system for organic agricultural product consumption scenarios based on multimodal perception, the system comprising an exhibition hall edge server, characterized in that, The exhibition hall edge server includes: One or more processors; A memory coupled to the one or more processors, wherein the one or more processors and the memory are configured as follows: The system receives a display collaboration request sent by a mobile terminal, wherein the display collaboration request includes at least a streaming media content source identifier, product information that the user is currently watching, and streaming media content playback status, wherein the display collaboration request is sent by the mobile terminal when it detects that the streaming media content playback status meets preset conditions, and wherein the streaming media content source identifier, the product information, and the streaming media content playback status are obtained by the mobile terminal during the user's viewing of streaming media content; The target video is determined based on the streaming media content source identifier and the product information; Determine whether multiple public display terminals are in an available state; and If it is determined that the first public display terminal among multiple public display terminals is in an available state, the target video is sent to the first public display terminal for playback, and the terminal identifier of the first public display terminal is sent to the mobile terminal to prompt the user about the first public display terminal.

7. The system according to claim 6, wherein, Based on the streaming media content source identifier and the product information, the target video is determined to include: Based on the streaming media content source identifier, a pre-stored streaming media content source association table is queried to determine whether the streaming media content source is a streaming media content source associated with the exhibition hall; When it is determined that the source of the streaming media content is a streaming media content source associated with the exhibition hall, the target video corresponding to the product information is retrieved from the video library based on the product information.

8. The system according to claim 6, wherein, The one or more processors and the memory are further configured to: If it is determined that multiple public display terminals are occupied, the display collaboration request is added to the playback request queue, and the multiple display collaboration requests in the playback request queue are sorted according to the quantized priority. Based on the sorting, the display collaboration requests that will be responded to first will be determined; The quantification priority is determined by the following factors: stuttering metric, request waiting time, number of requests for the same product within a preset statistical time window, and number of times the same mobile terminal has obtained public display terminal playback services within a preset time window.

9. The system according to claim 6, wherein, The one or more processors and the memory are further configured to: Before the exhibition hall edge server sends the target video to the first public display terminal, a playback time slot is allocated for the target video, and the first public display terminal is released when the exit conditions are met. The exit conditions include: the playback time slot expires and no continuation request is received from the mobile terminal.

10. The system according to claim 9, wherein, The playback time period is determined by the following factors: the remaining playback duration of the target video, the number of display collaboration requests currently waiting to use the public display terminal, the number of display collaboration requests for the same product within a preset statistical time window, and the number of times the same mobile terminal has obtained public display terminal playback services within a preset time window. Wherein, the remaining playback time of the target video is the length of time from the current playback position to the end position of the video; The number of display collaboration requests currently waiting to use the public display terminal is the number of display collaboration requests in the playback request queue that have not yet been allocated public display terminal playback resources when the playback occupation period is determined.

Images