Active scene intelligent processing method, device and equipment

Abstract

The invention relates to the field of scene intelligent processing, and discloses an active scene intelligent processing method, device and equipment, and the method comprises the steps: responding to a session initialization request, verifying an identity identifier carried by the session initialization request, and at least generating a session identifier and a verification identifier according to a verification result; in response to a session negotiation request generated by the session identifier and the verification identifier, session parameters are negotiated, and the session parameters at least comprise scene parameters; receiving scene data collected based on the session parameters, and preprocessing the scene data according to the scene parameters to obtain key plots and plot prompts of the scene data; and identifying the key plot and the plot prompt according to the scene parameters, and triggering a preset scene application according to an identification result. According to the technical scheme provided by one or more embodiments of the invention, active and efficient processing of the scene data can be realized.

Description

Technical Field

[0001] This application relates to the field of contextual intelligence processing, and in particular to an active contextual intelligence processing method, apparatus, and device. Background Technology

[0002] Contextual intelligence, as an application of artificial intelligence technology, collects visual data about the user's specific environment and situation, such as time, location, people, and activity status, and then performs intelligent analysis on the server side to provide users with corresponding application services.

[0003] In related technologies, contextual intelligence often relies on explicit user commands to initiate data processing flows, lacking the ability to proactively perceive environmental changes. Especially in situations requiring rapid response, it is difficult to dynamically adjust contextual applications based on the context, and the contextual data to be processed is often complex, resulting in low processing efficiency for contextual intelligence. Therefore, how to proactively and efficiently process contextual data has become a key research focus in the field of contextual intelligence. Summary of the Invention

[0004] This application provides a proactive contextual intelligence processing method, apparatus, and device that can achieve proactive and efficient processing of contextual data.

[0005] The first aspect of this application provides a proactive contextual intelligence processing method, the method comprising: in response to a session initialization request, verifying the identity identifier carried in the session initialization request, and generating at least a session identifier and a verification identifier based on the verification result; in response to a session negotiation request generated by the session identifier and the verification identifier, negotiating session parameters, the session parameters including at least contextual parameters; receiving contextual data collected based on the session parameters, and preprocessing the contextual data according to the contextual parameters to obtain key plots and plot hints of the contextual data; identifying the key plots and plot hints according to the contextual parameters, and triggering a preset contextual application according to the identification result.

[0006] In one implementation, in response to a session initialization request, verifying the identity carried in the session initialization request and generating at least a session identifier and a verification identifier based on the verification result includes: identifying the identity in the session initialization request, checking whether the identity is valid, and if the identity is valid, generating the session identifier and the verification identifier, wherein the session identifier is used to uniquely identify the current session and the verification identifier is used for authentication of subsequent requests; if the identity is invalid, generating an error message and terminating the current session.

[0007] In one implementation, the session negotiation request carries the session identifier, the verification identifier, and request parameters; in response to the session negotiation request generated by the session identifier and the verification identifier, the negotiated session parameters include: verifying the session identifier and verification identifier of the session negotiation request; if the verification is successful, confirming that the request parameters are session parameters and triggering a session response signal, the session response signal indicating that the session negotiation was successful.

[0008] In one embodiment, the method further includes: generating an error message and terminating the current session if the session identifier and the verification identifier fail to be verified.

[0009] In one implementation, the session parameters further include a context processing mode and frame processing parameters; in response to a session negotiation request generated by the session identifier and the verification identifier, negotiating the session parameters includes: determining the context processing mode based on the request parameters carried in the session negotiation request, wherein the context processing mode includes a static recognition mode and a dynamic recognition mode; and determining the frame processing parameters of the session negotiation request based on the request parameters carried in the session negotiation request, wherein the frame processing parameters include a video frame rate, a frame processing algorithm, and a total frame processing volume, wherein the frame processing parameters are used to characterize the frame processing method of the current session.

[0010] In one embodiment, the scenario parameters include scenario gaze parameters; preprocessing the scenario data according to the scenario parameters to obtain key plots and plot cues of the scenario data includes: processing the scenario data in terms of time dimension, spatial dimension, and semantic dimension according to the gaze dimensions and their combination order represented by the scenario gaze parameters, integrating the processed scenario data into key plots, and generating plot cues for the key plots.

[0011] In one embodiment, the preprocessing of the scenario data further includes: performing semantic dimension processing on the scenario data, and after integrating the semantically processed scenario data to obtain key plots, triggering scenario applications based on the key plots to achieve stress short-circuiting.

[0012] In one embodiment, the scenario parameters further include scenario identification parameters; identifying the key plot and the plot cue based on the scenario parameters includes: determining a fusion operator represented by the plot identification parameters, determining a target identification model for the key plot based on the fusion operator; inputting the key plot and the plot cue into the target identification model, and identifying the key plot and the key plot through the target identification model.

[0013] In one embodiment, the fusion operator includes a first operator, a second operator, and a third operator; determining the target identification model of the key plot based on the fusion operator includes: acquiring a preset identification model, the identification model including a multimodal large language model and a multimodal RAG (Retrieval-Augmented Generation) model, the multimodal large language model being used for reasoning about the key plot, and the multimodal RAG model being used for retrieving the key plot; when the fusion operator is the first operator, the multimodal large language model or the multimodal RAG model is determined as the target identification model; when the fusion operator is the second operator or the third operator, both the multimodal RAG model and the multimodal large language model are determined as the target identification model.

[0014] A second aspect of this application provides an active contextual intelligence processing device, the device comprising: an initialization unit, configured to, in response to a session initialization request, verify the identity identifier carried in the session initialization request, and generate at least a session identifier and a verification identifier based on the verification result; a session negotiation unit, configured to, in response to a session negotiation request generated by the session identifier and the verification identifier, negotiate session parameters, the session parameters including at least contextual parameters; a data acquisition and processing unit, configured to receive contextual data acquired based on the session parameters, and preprocess the contextual data according to the contextual parameters to obtain key plots and plot hints of the contextual data; and an event identification unit, configured to identify the key plots and plot hints according to the contextual parameters, and trigger a preset contextual application based on the identification result.

[0015] A third aspect of this application provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions, and the computer device is used to implement the proactive contextual intelligence processing method described in the first aspect.

[0016] The technical solutions provided in one or more embodiments of this application achieve efficient, flexible, and dynamic contextual intelligent services through dynamic parameter negotiation between the user end and the server end, key plot extraction, and intelligent event identification. Specifically, the server negotiates with the user end based on the session negotiation request, dynamically determines the context parameters to ensure the adaptability of data collection, improve the efficiency and accuracy of data collection, further refines key plots and generates plot prompts for key plots, reduces invalid data processing, improves the efficiency of contextual data processing, and flexibly and dynamically analyzes key plots based on the context parameters, dynamically makes decisions, and triggers corresponding operations.

[0017] As can be seen, the technical solution provided in this application can achieve proactive and efficient processing of contextual data. Furthermore, it can trigger corresponding contextual applications based on the processing results of different contextual data, enabling flexible adaptation to different intelligent scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of passive scene processing in a scene intelligence system provided in an embodiment of this application; Figure 2 A flowchart illustrating the steps of an active contextual intelligence processing method provided in this application. Figure 3(a) is a schematic diagram of the workflow of an identification model corresponding to a fusion operator provided in an embodiment of this application; Figure 3(b) is a schematic diagram of the workflow of an identification model corresponding to a fusion operator provided in another embodiment of this application; Figure 3(c) is a schematic diagram of the workflow of an identification model corresponding to a fusion operator provided in another embodiment of this application; Figure 4 A schematic diagram illustrating a dynamic recognition mode processing method of an HCAM mechanism provided in one embodiment of this application; Figure 5 A schematic diagram of an operation of a fusion operator based on the HCCM mechanism is provided for one embodiment of this application; Figure 6 A schematic diagram of the parameter interaction process for session initialization and session negotiation provided in one embodiment of this application; Figure 7 A schematic diagram of session transition in a scenario-based session state machine provided in one embodiment of this application; Figure 8 A schematic diagram of an active scene intelligent processing device provided in one embodiment of this application; Figure 9 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Furthermore, the use of terms such as "first," "second," etc., in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of embodiments in this application, unless otherwise stated, "multiple" means two or more. Additionally, the use of "based on" or "according to" implies openness and inclusiveness, because processes, steps, calculations, or other actions "based on" or "according to" one or more of the stated conditions or values ​​may in practice be based on additional conditions or beyond the stated values.

[0022] Contextual intelligence, as an application of artificial intelligence technology, has developed rapidly in recent years in fields such as computers, the internet, and information communication. Its core lies in collecting visual data about the user's specific environment and situation, including information such as time, location, people, and activity status, and then using the powerful intelligent analysis capabilities of the server to process this data, ultimately providing users with application services more tailored to their current context.

[0023] However, contextual intelligence in related technologies often has significant limitations. Please refer to... Figure 1 In existing models, contextual intelligence systems typically consist of a user terminal and a server terminal. The user terminal relies on input commands to initiate the data processing flow and sends input data to the server. The server responds to the input commands, processes the input data, and passively returns the processing results. This approach results in a certain startup delay, and the server must wait for commands, making it unable to anticipate data changes and lacking proactive perception and real-time feedback mechanisms. The process of responding to user commands can be described as inputting contextual data, reasoning based on pre-learned data, and outputting results as a response. This passive response mode makes it difficult for the system to adjust contextual applications in real time according to dynamic environmental changes, especially in scenarios requiring rapid response, such as traffic management, emergency response, and intelligent security. In these situations, contextual intelligence processing efficiency is low and cannot meet actual needs.

[0024] Furthermore, the related technologies also have shortcomings in data processing. During the data acquisition phase, there is a lack of adaptability to different user devices. Data collected by some devices may suffer from formatting errors, missing parameters, or insufficient accuracy, making it difficult to ensure that the collected data meets the requirements of scenario processing. Moreover, since scenario data includes information on numerous environmental factors, such as temperature, humidity, light intensity, and background noise, processing all collected data leads to low efficiency in scenario data processing, making it difficult to efficiently extract key information and further affecting the accuracy and timeliness of scenario intelligence.

[0025] In view of this, this application proposes one or more embodiments of an active contextual intelligence processing method, apparatus, and device, which can solve the above-mentioned problems and achieve active and efficient processing of contextual data. Simultaneously, it triggers corresponding contextual applications based on the processing results of different contextual data, achieving flexible adaptation to different contextual intelligence scenarios.

[0026] Please see Figure 2 One embodiment of this application provides an active contextual intelligence processing method, which is applied to a server and may include the following steps: S1: In response to the session initialization request, verify the identity carried in the session initialization request, and generate at least a session identifier and a verification identifier based on the verification result.

[0027] S3: In response to a session negotiation request generated by the session identifier and the verification identifier, negotiate session parameters, the session parameters including at least scenario parameters.

[0028] S5: Receive the scenario data collected based on the session parameters, and preprocess the scenario data according to the scenario parameters to obtain the key plots and plot hints of the scenario data.

[0029] S7: Identify the key plot points and plot prompts based on the scenario parameters, and trigger the preset scenario application based on the identification results.

[0030] The aforementioned session initialization request is initiated by the user client to the server to establish a new session. The user client is a device that the user can directly interact with, such as augmented reality (AR) glasses, smart glasses, mobile phones, IoT devices, androids, and autonomous vehicles. It is used to initiate requests, receive responses from the server, collect data (such as images and videos), and transmit the data to the server. The server is a server or system that provides services and resources, typically including server hardware, operating system, applications, and services. It is used to receive requests from the user client, process them, and return appropriate responses.

[0031] In this embodiment, the aforementioned session initialization request typically carries parameters identifying the user's terminal device, including identity identifier, device model, number of cameras, or number of video streams. This allows the server to generate a session identifier and a verification identifier based on the user's parameters, thereby verifying the legitimacy of the request and identifying the requester. Upon successful verification, the server generates a session identifier and a verification identifier for subsequent session management and authentication during requests or data transmission.

[0032] The aforementioned session negotiation request is generated based on a session identifier and a verification identifier, used to characterize the validity of the ongoing session. This request carries session parameters that define the specific configuration of the session, guiding the server and client on how to interact and process data. Typically, these session parameters are defined as a set of parameters, including multiple related parameters such as data format, transmission protocol, and processing algorithm, guiding data acquisition and processing during the session and ensuring that the client and server follow the same rules and configurations during data interaction. Among these session parameters, at least scenario parameters are included. These scenario parameters define configurations related to scenario data processing. For example, scenario parameters may define the type of scenario data (e.g., image, video), parameters of the acquisition device (e.g., resolution, frame rate), and data processing mode (e.g., static recognition, dynamic recognition).

[0033] In this implementation, the server determines specific session parameters through a session negotiation request. After establishing a session, the client sends a session negotiation request to the server to customize the session's behavior. After verifying the session identifier and the validity of the verification identifier, the server determines the final session parameters based on the parameter suggestions in the request and its own resource availability, providing the necessary configuration for subsequent data processing.

[0034] The aforementioned contextual data is collected based on session parameters and can be in the form of images, videos, etc., to reflect the user's specific environment and situation. The aforementioned key plot points can be understood as the key parts extracted from the contextual data, used to characterize the core content to be analyzed in the entire context. The aforementioned plot cues can be understood as auxiliary information extracted from the contextual data, used to provide additional context or guidance, enabling a more accurate understanding and analysis of the key plot points.

[0035] In this embodiment, key plot points and plot cues are extracted from the acquired scenario data for subsequent data analysis. Specifically, the scenario data is first verified to meet the requirements of the session parameters. The scenario data is then preprocessed, including but not limited to format conversion, data cleaning, and feature extraction. Key plot points and plot cues are extracted from the preprocessed data for subsequent intelligent scenario analysis, such as event recognition and decision-making. This allows the server to more quickly understand and respond to the user's situation, avoiding interference from some noisy data in the scenario data and ensuring the accuracy and efficiency of subsequent intelligent analysis.

[0036] In this embodiment, the server triggers corresponding preset scenario applications based on the identification results of key plots to respond to and process scenario events. These scenario applications are predefined according to the needs of the scenario events and are used to respond to specific scenario events, such as sending notifications or executing automated tasks. Specifically, the server receives preprocessed scenario data, including key plots and plot cues, and further identifies the key plots and cues using previously negotiated scenario parameters. This identification process can employ multimodal large language models to analyze and judge the key plots and cues, such as pattern recognition and semantic understanding, to determine the specific scenario event represented by the key plots and cues. When a key plot matches a preset scenario event, the server directly triggers the corresponding scenario application, achieving proactive and efficient processing of scenario data, thereby enabling accurate response and efficient processing of scenario events.

[0037] In one implementation, the identity identifier carried in the session initialization request is verified before initiating a scenario session to ensure the security and legitimacy of the session. Specifically, after receiving the session initialization request from the user, the server identifies the identity identifier in the request, which can be a username, password, device ID, etc. The legitimacy of the identity identifier can be determined by matching it against a preset user database; if a match is found, the current identity identifier is deemed legitimate, preventing unauthorized access. Alternatively, the identity identifier can be checked to ensure it conforms to preset formats and rules; if it does, the current identity identifier is deemed legitimate, ensuring a secure and reliable session connection is established between the user and the server.

[0038] Furthermore, if the identity is deemed valid, the server generates a session identifier and a verification identifier. The session identifier uniquely identifies the current session, while the verification identifier is used for authentication of subsequent requests, verifying the user's identity during the session. If the identity is invalid, the server generates an error message and returns it to the user, informing them of the reason for the request failure and terminating the current session. This clear error message and session management mechanism ensures that users can successfully perform legitimate operations.

[0039] The technical solution provided in this embodiment details the process of how the server handles session initialization requests. The server verifies the identity identifier in the request to determine whether to generate a session identifier and a verification identifier, or to return an error message and terminate the session. Verifying the identity identifier ensures the security and legitimacy of the session, while effective session management and authentication improve the overall security of the system and the user experience.

[0040] In one implementation, if the session identifier and verification identifier in the session negotiation request are valid, session parameters are negotiated. Specifically, the session negotiation request carries a session identifier, a verification identifier, and request parameters. In response to the session negotiation request generated by the session identifier and verification identifier, the session parameters are negotiated by first verifying the session identifier and verification identifier in the session negotiation request.

[0041] In this implementation, if the session identifier and verification identifier pass verification, the requested parameters are confirmed as session parameters, ensuring that both parties agree on the session configuration and improving the accuracy and efficiency of session management. Simultaneously, a session response signal is triggered, indicating successful session negotiation. The aforementioned requested parameters are the initial parameters carried by the user in the session negotiation request, used to provide the specific configuration of the session. The aforementioned session parameters are the configuration parameters determined through negotiation, guiding data interaction and processing during the session. The aforementioned session response signal is a signal sent by the server to the user, ensuring that the user knows that the negotiation has been successfully completed.

[0042] In this implementation, if the session identifier and verification identifier fail to verify, an error message is generated and the current session is terminated. Specifically, the server generates an error message and returns it to the client to inform the client of the reason for the request failure and proactively closes the current session to prevent further unauthorized access and data leakage. This ensures that only legitimate sessions can continue, improving the reliability and stability of the system.

[0043] The technical solution provided in this embodiment achieves effective management of session parameter negotiation by verifying the session identifier and verification identifier in the session negotiation request. When the identifier verification passes, the request parameters carried by the user are confirmed as session parameters, ensuring that both parties reach an agreement on the session configuration. At the same time, a session response signal is triggered to inform the user that the negotiation has succeeded, improving the accuracy and efficiency of session management. When the identifier verification fails, the server generates an error message and sends it back to the user, explaining the reason for the request failure, and actively closes the current session, thereby preventing unauthorized access and data leakage, ensuring that only legitimate sessions can continue, and enhancing the reliability and stability of the system.

[0044] In one implementation, the session parameters further include a context processing mode and frame processing parameters, which are negotiated in response to a session negotiation request. Specifically, the context processing mode is determined based on the request parameters carried in the session negotiation request, and the context processing mode includes a static recognition mode and a dynamic recognition mode. Furthermore, the frame processing parameters of the session negotiation request are determined based on the request parameters carried in the session negotiation request, and the frame processing parameters include the video frame rate, frame processing algorithm, total frame processing volume, etc., and these frame processing parameters characterize the frame processing method of the current session.

[0045] The aforementioned scenario processing modes indicate the overall processing strategy of the session for scenario data, used to limit the server to perform static recognition on a single frame or dynamic recognition on a continuous sequence of frames. These scenario processing modes are directly specified by the request parameters carried in the session negotiation request. The scenario processing modes include static recognition mode and dynamic recognition mode. The static recognition mode can be understood as processing only a single frame or a fixed number of discrete images, without relying on continuous temporal information. The dynamic recognition mode can be understood as processing a continuous sequence of frames, using temporal correlation to complete the recognition task. These scenario processing modes are explicitly given by the request parameters carried in the session negotiation request. The aforementioned frame processing parameters specifically characterize the session's specific processing method for video frames. By specifying the video frame rate, frame processing algorithm, and total number of frames processed, the frame-level operation rules of the server during acquisition, sampling, and calculation are defined.

[0046] The technical solution provided in this embodiment achieves more granular session configuration by extending session parameters to context processing modes and frame processing parameters, and negotiating both in response to session negotiation requests. On the one hand, determining the context processing mode based on the request parameters clarifies the overall processing strategy of the session for context data and limits the server's identification method. On the other hand, determining the frame processing parameters based on the request parameters clarifies the specific processing method of the session for video frames and limits the server's operating rules in the frame acquisition, sampling, and calculation processes. This makes session processing more aligned with actual needs and improves the relevance and accuracy of context data processing.

[0047] In one implementation, the context parameters include context gaze parameters. The context data is preprocessed based on these context gaze parameters to obtain key plot points and plot cues. Specifically, the context data is processed in terms of time dimension, spatial dimension, and semantic dimension according to the gaze dimensions and gaze combination order represented by the context gaze parameters. The processed context data is then integrated into key plot points, and plot cues for these key plot points are generated.

[0048] The aforementioned contextual fixation parameters characterize the fixation dimensions and the fixation combination order, and can be represented by multiple parameters. These parameters may include fixation dimension parameters and dimension combination method parameters. The fixation dimension parameters characterize the dimensions of interest when processing contextual data, including temporal, spatial, and semantic dimensions. The dimension combination method parameters characterize the combination method of fixation dimensions and their combination order. These combination methods include sequential and parallel combinations; for example, processing the contextual data temporally first, then processing the temporally processed data spatially, or simultaneously processing the contextual data temporally and spatially.

[0049] For example, the aforementioned temporal dimension processing can be used to remove duplicate frames and retain significantly changing frames; the aforementioned spatial dimension processing can be used to segment regions focusing on ROIs (Regions of Interest) or specific targets; and the aforementioned semantic dimension processing can be understood as capturing sudden events or significant semantic labels. Furthermore, the contextual data is processed according to the gaze dimension and gaze combination order to obtain key plots and plot cues extracted from the key data. The key plots after different dimension processing can be in the form of individual frames, regions, or label groups, or in the form of an integrated set of merged frames, regions, or label groups. The aforementioned plot cues can be in the form of dimension sources, timestamps, or labels, either individually or in combination.

[0050] The technical solution provided in this embodiment achieves accurate extraction of key plot points and plot cues by introducing contextual gaze parameters and preprocessing contextual data accordingly. Specifically, according to the gaze dimensions and combination order specified by the contextual gaze parameters, the contextual data is processed in terms of time dimension, spatial dimension, and / or semantic dimension. The processed data is then integrated into key plot points and plot cues are generated. Through clear dimensional division and combination rules, targeted screening and integration of contextual data are achieved, ensuring that the extracted key plot points focus on core information and that the plot cues are clear to assist subsequent identification. This improves the effectiveness and accuracy of contextual data preprocessing and lays a reliable foundation for subsequent response processing of contextual events.

[0051] In one implementation, after semantic dimension processing of the scenario data and integration of the semantically processed scenario data to obtain key events, scenario applications are triggered based on the key events. Specifically, the semantic dimension processing can be based on an event classifier or an anomaly detection model, outputting the input raw scenario data as event labels with semantic scores. Key events are determined based on a preset threshold of the semantic scores, and scenario applications corresponding to the key events are triggered, such as preset actions like alarms, emergency recording, and flashing lights. After triggering, the key events are cached for later use in logs or reviews.

[0052] The technical solution provided in this embodiment processes scenario data semantically and determines key stress points based on a preset threshold of semantic scores. This allows for the direct triggering of corresponding scenario applications, enabling rapid response to emergency or special events. Furthermore, the key stress points are cached after triggering, providing data support for subsequent log recording and post-mortem analysis. This improves the timeliness and relevance of scenario event handling while ensuring process traceability.

[0053] In one implementation, the scenario parameters further include scenario identification parameters, which are used to identify key plot points and plot cues. Specifically, a fusion operator representing the plot identification parameters is determined, and a target identification model for key plot points is determined based on the fusion operator; the key plot points and plot cues are input into the target identification model, and the key plot points and plot cues are identified through the target identification model.

[0054] The aforementioned scenario identification parameters, as session-level configuration items, include multiple configuration parameters such as fusion operators. The number and type of fusion operators can be determined based on the scenario identification parameters. These fusion operators characterize the model processing method for key plots and are used to guide model selection and fusion logic in the event identification stage. The aforementioned target identification model can be understood as the actual execution unit mapped by the fusion operators. It can be a single identification model or a combination of multiple models defined by operators. It is used to receive key plots and plot cues and output event categories to trigger corresponding preset scenario applications.

[0055] In this embodiment, the fusion operator includes a first operator, a second operator, and a third operator. The target identification model for key plots is determined based on the fusion operator. The fusion operator can be composed of three mutually exclusive enumeration values, sequentially mapped to the call path of the target identification model. The first operator '∃' indicates triggering upon single-path satisfaction, the second operator '∥' indicates pipelined concatenation, and the third operator '∩' indicates simultaneous satisfaction of both paths. The identification model includes a multimodal large language model and a multimodal RAG model. The multimodal large language model is used to reason about key plots, generating inference results such as logical judgments, event categories, and confidence levels. The multimodal RAG model is used to retrieve key plots.

[0056] Specifically, please refer to Figure 3(a). When the fusion operator is the first operator, either the multimodal large language model or the multimodal RAG model is determined as the target identification model. Either one can trigger the corresponding subsequent operation once it completes the identification of the contextual event. Please refer to Figure 3(b). When the fusion operator is the second operator, both the multimodal RAG model and the multimodal large language model are determined as target identification models. The result of the multimodal RAG model processing the contextual data is then output to the multimodal large language model for further discrimination. After sequential processing, the corresponding subsequent operation is triggered. Please refer to Figure 3(c). When the fusion operator is the third operator, both the multimodal RAG model and the multimodal large language model are determined as target identification models. Both the multimodal large language model and the multimodal RAG model must complete the identification of the contextual event before the corresponding subsequent operation can be triggered.

[0057] The technical solution provided in this embodiment clarifies the model processing method for key plots in the fusion operator of the scenario identification parameters. The reasoning ability of the multimodal large language model is combined with the retrieval ability of the multimodal RAG model, each leveraging its strengths under different fusion logics. This allows the identification of key plots to balance efficiency, depth, and accuracy, providing a precise basis for subsequently triggering corresponding preset scenario applications.

[0058] This application provides an embodiment proposing an HCAM (HyperContextAttention Mechanism) mechanism for preprocessing contextual data and an HCCM (HyperContextCognition Mechanism) mechanism for intelligently distinguishing contextual events in visual information. The HCAM mechanism can refine contextual data across three dimensions based on methods such as object detection, instance segmentation, object tracking, similarity comparison, and sentiment analysis, and arbitrarily arrange the priority and parallel order of the three dimensions. Key plots extracted by the HCAM mechanism in the semantic dimension can directly trigger subsequent application operations, achieving stress-induced short-circuiting of the HCCM mechanism's contextual discrimination. Simultaneously, the HCAM mechanism can also use the refined key plots and plot cues as inputs to the HCCM mechanism, influencing the discrimination of contextual events. Furthermore, the HCAM mechanism can provide feedback on the interaction relationship between the three dimensions based on key plots and plot cues, thereby achieving dynamic adjustment of the contextual attention mechanism.

[0059] In this embodiment, please refer to Figure 4 , Figure 4The workflow of the HCAM mechanism is demonstrated, using frame extraction as an example. In dynamic recognition mode, the HCAM mechanism is applied to sample consecutive frames. The contextual data first undergoes multi-dimensional combination processing, followed by analysis through three steps: time-series comparison, spatial region focusing, and semantic information capture. These steps include object detection, instance segmentation, object tracking, similarity comparison, and sentiment analysis. These three steps correspond to the extraction of the temporal, spatial, and semantic dimensions of the contextual data, respectively. The processed and refined contextual data undergoes feedback adjustment to optimize the extraction of key elements. In dynamic recognition mode, the HCAM mechanism can sample consecutive frames. The frame processing algorithm is specified by the user during session negotiation, including types such as approximate sampling and uniform sampling, as well as the number of sampled frames. Uniform sampling samples input image frames at fixed intervals, while approximate sampling samples based on the similarity distribution of image frames to reduce the sampling frequency in time-series ranges with excessively high similarity between consecutive frames, avoiding the collection of too many similar images and thus improving the efficiency of multimodal language large-scale model inference. Unlike dynamic recognition mode, static recognition mode only extracts one frame from consecutive frames for processing.

[0060] In this embodiment, further please refer to Figure 5 The application, after image preprocessing, uses the fusion operator in the HCCM mechanism to intelligently distinguish contextual events in visual information. The HCAM mechanism outputs key plot points and plot cues, which, along with system and identity cues, are injected into the HCCM operator. The HCCM operator receives these inputs and performs contextual retrieval using MRAG (Multimodal RAG Model) and contextual inference using MLLM (Multimodal Large Language Model), respectively. Taking the ∃ fusion operator as an example, the HCCM mechanism receives key plot points and plot cues from the HCAM output. At this point, the HCCM mechanism has already pre-injected system cues and terminal-related identity cues. The HCCM mechanism fuses the system cues, identity cues, and plot cues, performing contextual retrieval using MRAG and contextual inference using MLLM, respectively. When either the MRAG or MLLM return value satisfies the logical judgment, the HCCM mechanism will trigger personalized application services based on the identity profile.

[0061] Please refer to Figure 6 This application also provides embodiments for one or more requests. Figure 6The above embodiments describe the request processing flow of the HCTP (HyperContextTransfer Protocol). The above embodiments focus on the parameter interaction flow of session initialization and session negotiation based on the HCTP protocol. In these embodiments, session initialization, session negotiation, context identification, and context session termination are performed based on the HCTP protocol. In these steps, the request body parameters and return body parameters of each request in the HCTP protocol are used to characterize the parameter interaction between the user end and the server end. Specific parameters used can be found in Tables 1, 2, 3, and 4. Table 1. Request body parameters and return body parameters for the session initialization process: Table 2. Request body parameters and return body parameters of the session negotiation process: Table 3. Request body parameters and return body parameters of the scene recognition process: Table 4. Request body parameters and return body parameters for the scenario conversation termination process: In one embodiment, please refer to Table 1 above and Figure 6 The session initialization request carries a set of request body parameters, including multiple parameters such as the identity identifier. The request body parameters of the above session initialization request can be represented as {identity, device, cnum, vnum, kargs}, where 'identity' is the identity identifier, used by the session management module to verify the identity of the requester, 'device' is the device model, used to identify the terminal device initiating the request, 'cnum' is the number of cameras, used to indicate the number of cameras on the terminal device, 'vnum' is the number of video streams, used to indicate the number of video streams on the terminal device, and 'kargs' is an extensible parameter, used to pass additional, optional parameters.

[0062] In this embodiment, after receiving the session initialization request and completing authentication, the server sends a session response signal to the client. The session response signal carries a set of return body parameters, including a session identifier and an authentication identifier. These return body parameters can be represented as {contextID, contextToken, model, sample, res, mllm, mRAG, kargs}, where 'contextID' is the session identifier, used to uniquely identify a session; 'contextToken' is the authentication identifier, used for authentication of subsequent requests; 'model' is the context processing mode, used to indicate how the application processes context data; 'sample' is the frame processing algorithm, used to indicate how to process video frames; 'res' is the required terminal capture resolution, used to indicate the video resolution the terminal should capture; 'mllm' is a list of multimodal large language models supported by the backend, used to indicate the multimodal large language models available by the backend; 'mRAG' is a list of multimodal RAGs supported by the backend, used to indicate the multimodal RAGs available by the backend; and 'kargs' is an extensible parameter used to return additional, optional information.

[0063] In one embodiment, please refer to Table 2 above and Figure 6 The session negotiation request carries a set of request body parameters, including multiple parameters such as a session identifier and a verification identifier. These request body parameters can be represented as {contextID, contextToken, model, fps, sample, frames, mllm, mRAG, dims, cb, operator, eoc, krags}, where 'contextID' is the session identifier, identifying the ongoing session; 'contextToken' is the verification identifier, used to verify the validity of the session; 'model' is the context processing mode, used to instruct the application how to process context data; 'fps' is the video frame rate, used to indicate the frame rate of the video stream; 'samp' is the frame rate, and 'mr ... 'le' is the frame processing algorithm, indicating how to process video frames; 'frames' is the total number of frames to process; 'mllm' is the multimodal large language model selected after application negotiation; 'mRAG' is the multimodal RAG selected after application negotiation; 'dims' is the gaze dimension; 'cb' is the gaze combination order; 'operator' is the fusion operator; 'eoc' is the session closure condition; 'kargs' are extensible parameters used to pass additional, optional parameters.

[0064] In this embodiment, after the server receives the information carried in the session negotiation request and completes the negotiation, it sends a session response signal to the user. The session response signal carries a return body parameter, such as the 'ACK' parameter, to confirm that the session negotiation was successful.

[0065] In one embodiment, please refer to Table 3 above and Figure 6 The context data 'data' is sent to the server as a request body parameter for data processing. The request body parameters include 'contextID', which identifies the ongoing session, 'contextToken', which verifies the validity of the session, and 'kargs', which are extensible parameters used to pass additional, optional parameters.

[0066] In this embodiment, after receiving the request body parameters, the server processes the scenario data according to the scenario parameters and sends a response signal to the user. The response signal carries a return body parameter, such as 'result', which is used to characterize the scenario processing result.

[0067] In one embodiment, please refer to Table 4 above and Figure 6 The client sends a request with request body parameters to the server to notify the server to terminate the session. The request body parameters include 'contextID', which identifies the ongoing session, i.e. the session that needs to be terminated, and 'contextToken', which is used to verify the validity of the session.

[0068] In one embodiment, see Figure 7 , Figure 7 The session transition process is characterized by a contextual session state machine. The contextual session state machine is used to represent the state of the contextual session, including "session authentication," "session negotiation," "session persistence," and "session closure." When the terminal initiates and completes authentication, the contextual session state is "session authentication." Then, the contextual parameter negotiation phase begins. If authentication is successful and negotiation is completed, the contextual session state is "session negotiation"; if authentication fails, the contextual session state changes to "session closure." Next, the terminal sends contextual data. If the HCAM mechanism does not capture the stress event and the HCCM mechanism does not identify the service event, the contextual session state is "session persistence." Conversely, if the HCAM mechanism triggers an application under stress or the HCCM service triggers an application, the contextual session is closed. Alternatively, the terminal can also proactively initiate session closure. After the session is closed, the terminal can re-initiate the session, at which point the contextual session state changes back to "session authentication."

[0069] This application provides one or more embodiments of the above-described active scene intelligence processing method, using AR (Augmented Reality) glasses as the user terminal and a scenario where a session request for services from the user terminal is initiated to the server. The server performs static or dynamic scene recognition, and the following embodiments provide a detailed description: Example 1: The AR glasses user initiates a static scene recognition request and uses a multimodal large language model for inference.

[0070] Specifically, the AR glasses user initiates a session negotiation request with specified parameters to the server. These parameters include: one camera, two video streams, static recognition mode, 448*448 resolution, time-based gaze dimension, approximate sampling for frame processing, a multimodal large language model (e.g., qwen-2.5-Omni-7B) for the recognition backend, and a custom termination condition. After negotiation, the AR glasses send image frames to the server according to the above agreement. The server extracts the images from the user's images along the time dimension and combines the extracted images with system prompts. The multimodal large language model analyzes and infers from the integrated images and prompts. When the session termination condition is met, the server optimizes the output format of the inference results and returns the recognition results to the user. Furthermore, the AR glasses can explicitly terminate the session at any time. In this case, the user will stop sending data, and the multimodal large language model will simultaneously terminate the inference process.

[0071] Example 2: When the AR glasses user initiates a static scene recognition request, multimodal RAG is used for retrieval.

[0072] Specifically, the AR glasses user initiates a session negotiation request with specified parameters to the server. These parameters include: one camera, one video stream, static recognition mode, 1280*720 resolution, spatial gaze dimension, a multimodal RAG model (e.g., OmniSearch) for the recognition backend, and a custom termination condition. After negotiation, the AR glasses send image frames to the server according to the above agreement. The server captures key details from the user's images in the spatial dimension, extracts them, and combines them with system prompts. The multimodal RAG model analyzes and retrieves the integrated images and prompts. When the session termination condition is met, the server optimizes the search results and returns the parsed output to the AR glasses. Furthermore, the AR glasses can explicitly terminate the session at any time by requesting the session module. In this case, the user will stop sending data, and the multimodal RAG will simultaneously terminate the search process.

[0073] Example 3: When the AR glasses user initiates a static scene recognition request, multimodal RAG and multimodal large language model are used for collaborative processing.

[0074] Specifically, the AR glasses user initiates a session negotiation request with specified parameters to the server. These parameters include: one camera, one video stream, static recognition mode, 360*640 resolution, a parallel combination of temporal and spatial gaze dimensions, a multimodal RAG (e.g., OmniSearch) and a multimodal large language model (e.g., qwen-2.5-Omni-7B) for the recognition backend, a pipelined concatenation operator (∥), and a custom termination condition. After negotiation, the AR glasses send image frames to the server according to the above agreement. The server simultaneously extracts temporal and spatial dimensions from the images sent by the terminal, and combines the extracted images with system prompts. The integrated image and prompts are first retrieved by the multimodal RAG, and the retrieval results are then input into the multimodal large language model for further inference. When the session termination condition is met, the server optimizes the inference results and returns the parsed output to the AR glasses. In addition, AR glasses can explicitly request to terminate the session at any time. At this time, the user will stop sending data, and the multimodal large language model will also terminate the reasoning process simultaneously.

[0075] Example 4: The AR glasses client initiates a dynamic scene recognition request.

[0076] Specifically, the AR glasses user initiates a session negotiation request with specified parameters to the server. These parameters include: one camera, one video stream, dynamic recognition mode, 448*448 resolution, 3fps frame rate, 64 frames processed, a gaze dimension combining temporal, spatial, and semantic dimensions in parallel, a multimodal large language model (e.g., qwen-2.5-Omni-7B) for the recognition backend, and a custom termination condition. After negotiation, the AR glasses send image frames to the server according to the above agreement. The server continuously extracts temporal, spatial, and semantic dimensions from the received consecutive image frames, keeping the total number of frames input to the multimodal large language model at 64. The multimodal large language model analyzes and infers from the consecutive images and prompts. When the session termination condition is met, the server optimizes the output format of the inference result and then returns the parsed output to the AR glasses. In addition, AR glasses can explicitly request to terminate the session at any time. At this time, the user will stop sending data, and the multimodal large language model will also terminate the reasoning process simultaneously.

[0077] Please see Figure 8This application also provides an active contextual intelligence processing device, the device comprising: Initialization unit 100 is used to respond to a session initialization request, verify the identity identifier carried in the session initialization request, and generate at least a session identifier and a verification identifier based on the verification result; The session negotiation unit 200 is configured to negotiate session parameters in response to a session negotiation request generated by the session identifier and the verification identifier, wherein the session parameters include at least scenario parameters; The data acquisition and processing unit 300 is used to receive scenario data acquired based on the session parameters, and preprocess the scenario data according to the scenario parameters to obtain the key plots and plot hints of the scenario data. The event identification unit 400 is used to identify the key plot and the plot prompt based on the scenario parameters, and to trigger a preset scenario application based on the identification result.

[0078] in, In one implementation, the initialization unit 100 is specifically used to identify the identity identifier in the session initialization request, check whether the identity identifier is valid, and if the identity identifier is valid, generate the session identifier and the verification identifier. The session identifier is used to uniquely identify the current session, and the verification identifier is used for authentication of subsequent requests. If the identity identifier is invalid, generate an error message and terminate the current session.

[0079] In one implementation, the session negotiation unit 200 is specifically used to verify the session identifier and verification identifier of the session negotiation request. If the verification is successful, the request parameter is confirmed as a session parameter, and a session response signal is triggered. The session response signal indicates that the session negotiation is successful. If the verification of the session identifier and the verification identifier fails, an error message is generated and the current session is terminated.

[0080] In one embodiment, the session negotiation unit 200 is further configured to determine the scenario processing mode based on the request parameters carried in the session negotiation request, the scenario processing mode including a static identification mode and a dynamic identification mode, and to determine the frame processing parameters of the session negotiation request based on the request parameters carried in the session negotiation request, the frame processing parameters including video frame rate, frame processing algorithm, and total frame processing volume, the frame processing parameters being used to characterize the frame processing method of the current session.

[0081] In one embodiment, the acquisition and processing unit 300 is specifically used to process the scenario data in terms of time dimension, space dimension and / or semantic dimension according to the gaze dimension and gaze combination order represented by the scenario gaze parameters, integrate the processed scenario data into key plots, generate plot prompts for the key plots, and after processing the scenario data in terms of semantic dimension and integrating the key plots based on the semantically processed scenario data, trigger the scenario application according to the key plots.

[0082] In one embodiment, the event identification unit 400 is specifically used to determine the fusion operator representing the plot identification parameters, including a first operator, a second operator, and a third operator; determine the target identification model of the key plot based on the fusion operator, including a multimodal large language model and a multimodal RAG model; input the key plot and the plot cues into the target identification model; and identify the key plot and the plot cues through the target identification model.

[0083] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0084] An active scene intelligence processing device in this application embodiment is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, or other devices that can provide the above functions.

[0085] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application, such as... Figure 9 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 9 Take a processor 10 as an example.

[0086] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0087] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0088] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0089] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0090] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0091] The apparatus, module, or unit described in the above embodiments can be implemented by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0092] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0093] Those skilled in the art will understand that embodiments of this application can be provided as methods or systems. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and devices according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0095] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0096] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0097] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0098] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0099] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

[0100] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A proactive contextual intelligence processing method, characterized in that, The method includes: In response to a session initialization request, verify the identity carried in the session initialization request, and generate at least a session identifier and a verification identifier based on the verification result; In response to a session negotiation request generated by the session identifier and the verification identifier, session parameters are negotiated, the session parameters including at least scenario parameters; Receive scenario data collected based on the session parameters, and preprocess the scenario data according to the scenario parameters to obtain the key plots and plot hints of the scenario data; Based on the scenario parameters, the key plot points and plot prompts are identified, and a preset scenario application is triggered based on the identification results.

2. The method according to claim 1, characterized in that, In response to a session initialization request, the identity identifier carried in the session initialization request is verified, and at least a session identifier and a verification identifier are generated based on the verification result, including: Identify the identity identifier in the session initialization request, check whether the identity identifier is valid, and if the identity identifier is valid, generate the session identifier and the verification identifier. The session identifier is used to uniquely identify the current session, and the verification identifier is used for authentication of subsequent requests. If the identity is invalid, an error message is generated and the current session is terminated.

3. The method according to claim 1, characterized in that, The session negotiation request carries the session identifier, the verification identifier, and request parameters; in response to the session negotiation request generated by the session identifier and the verification identifier, the negotiated session parameters include: Verify the session identifier and verification identifier of the session negotiation request. If the verification is successful, confirm that the request parameter is a session parameter and trigger a session response signal. The session response signal indicates that the session negotiation was successful.

4. The method according to claim 1 or 3, characterized in that, The session parameters also include a context processing mode and frame processing parameters; in response to a session negotiation request generated by the session identifier and the verification identifier, the session parameters are negotiated, including: The scenario processing mode is determined based on the request parameters carried in the session negotiation request. The scenario processing mode includes a static identification mode and a dynamic identification mode. The frame processing parameters of the session negotiation request are determined based on the request parameters carried in the session negotiation request. The frame processing parameters are used to characterize the frame processing method of the current session.

5. The method according to claim 1, characterized in that, The scenario parameters include scenario gaze parameters; the scenario data is preprocessed based on the scenario parameters to obtain key plot points and plot cues, including: According to the gaze dimension and gaze combination order represented by the context gaze parameters, the context data is processed in terms of time dimension, space dimension, and semantic dimension. The processed context data is integrated into key plots, and plot cues for the key plots are generated.

6. The method according to claim 1 or 5, characterized in that, Preprocessing the scenario data further includes: After semantic dimension processing of the scenario data and integration of the semantically processed scenario data to obtain key plots, the scenario application is triggered according to the key plots.

7. The method according to claim 1, characterized in that, The scenario parameters also include scenario identification parameters; identifying the key plot points and plot cues based on the scenario parameters includes: Determine the fusion operator representing the plot identification parameters, and determine the target identification model of the key plot based on the fusion operator; The key plot points and plot hints are input into the target identification model, and the target identification model identifies the key plot points and plot hints.

8. The method according to claim 7, characterized in that, The fusion operator includes a first operator, a second operator, and a third operator; the target identification model for determining the key plot based on the fusion operator includes: A preset identification model is obtained, which includes a multimodal large language model and a multimodal RAG model. The multimodal large language model is used to reason about the key plot, and the multimodal RAG model is used to retrieve the key plot. When the fusion operator is the first operator, the multimodal large language model or the multimodal RAG model is determined as the target recognition model; When the fusion operator is the second operator or the third operator, both the multimodal RAG model and the multimodal large language model are determined as target recognition models.

9. An active contextual intelligent processing device, characterized in that, The device includes: An initialization unit is used to respond to a session initialization request, verify the identity identifier carried in the session initialization request, and generate at least a session identifier and a verification identifier based on the verification result; A session negotiation unit is configured to negotiate session parameters in response to a session negotiation request generated by the session identifier and the verification identifier, wherein the session parameters include at least scenario parameters; The data acquisition and processing unit is used to receive the scenario data acquired based on the session parameters, and to preprocess the scenario data according to the scenario parameters to obtain the key plots and plot hints of the scenario data. The event identification unit is used to identify the key plot and the plot prompt based on the scenario parameters, and to trigger a preset scenario application based on the identification result.

10. A computer device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the active contextual intelligence processing method according to any one of claims 1 to 8 by executing the computer instructions.

Images