Specified video monitoring and playing method and device based on intelligent agent, and storage medium

Abstract

The invention relates to an agent-based specified video monitoring and playing method and device and a storage medium. The method comprises the following steps: generating an instruction request in response to an operation of a user at a front end; when the instruction request is to call the live broadcast picture, generating a tool call link rule according to the instruction request in the live broadcast picture real-time streaming agent; and according to the tool calling link rule and the instruction request, calling corresponding tools in sequence to obtain a video stream address of the target monitoring equipment so as to load a corresponding live broadcast picture at the front end according to the video stream address. According to the application, the video stream address of the target monitoring device can be obtained through the corresponding agent calling tool according to the instruction request of the user at the front end, the live broadcast picture of the monitoring device specified by the user can be played more conveniently and efficiently, and the problem of low live broadcast picture searching efficiency in video monitoring is solved.

Description

Technical Field

[0001] The present application relates to the field of video surveillance technology, and in particular to a method, device and storage medium for designated video surveillance playback based on an intelligent agent. Background Art

[0002] In daily monitoring management, facility inspections and other work, staff need to frequently check camera images at different locations. Currently, in the AI ​​(Artificial Intelligence) analysis application scenarios for camera devices, the location of camera devices clearly has a geographical hierarchy of provinces, cities, districts, streets, etc., which provides a certain logical framework for the management and positioning of devices.

[0003] When it is necessary to view the live broadcast image of a specific camera, the existing search method is to gradually search for the target device according to the geographical level. For example, start screening from the provincial level, and then gradually go deeper to the city level, district level, and street level, and finally locate the specific camera device. This method is extremely cumbersome and requires a lot of time and effort. Especially when the customer has a large number of camera devices or in an emergency, the efficiency is extremely low, which seriously affects work efficiency and user experience, and also restricts the efficient development of related businesses.

[0004] With regard to the problem of low efficiency in searching live images in video surveillance in related technologies, no effective solution has been proposed so far. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device and storage medium for quickly acquiring the required live broadcast images based on an intelligent agent in order to solve the above technical problems.

[0006] In the first aspect, in this embodiment, a method for monitoring and playing a designated video based on an agent is provided, comprising: In response to the user's operation on the front end, generating an instruction request; When the instruction request is to call a live screen, in the live screen real-time stream agent, a tool call link rule is generated according to the instruction request; According to the tool calling link rule and the instruction request, the corresponding tools are called in sequence to obtain the video stream address of the target monitoring device, so as to load the corresponding live screen according to the video stream address at the front end.

[0007] In some embodiments, after generating the instruction request, the method further includes: Analyzing semantic information of the command request through an intent recognition agent; According to the semantic information, the instruction request is forwarded to the relevant agent for processing.

[0008] In some embodiments, generating a tool call link rule according to the instruction request includes: Determine the required tool according to the instruction request, and correspondingly generate the tool calling link rule including the tool calling sequence; The tool includes a first tool for obtaining a device code of the target monitoring device, and a second tool for obtaining a video stream address.

[0009] In some embodiments, determining the required tool according to the instruction request and correspondingly generating the tool call link rule including the call sequence includes: When the instruction request includes a device code, generating a tool call link rule including the second tool; When the instruction request does not include a device code, a tool call link rule is generated that includes the first tool and the second tool in sequence.

[0010] In some embodiments, generating an instruction request in response to an operation of a user on the front end includes: In response to the text input or voice input of the user, an instruction request is generated; the instruction request is a natural language instruction.

[0011] In some of the embodiments, calling corresponding tools in sequence to obtain the video stream address of the target monitoring device according to the tool calling link rule and the instruction request includes: The device code in the instruction request is input into the second tool to obtain the video stream address of the target monitoring device.

[0012] In some of the embodiments, calling corresponding tools in sequence to obtain the video stream address of the target monitoring device according to the tool calling link rule and the instruction request includes: Inputting the device name or device location in the instruction request into the first tool to obtain the device code of the target monitoring device; The device code is input into the second tool to obtain the video stream address of the target monitoring device.

[0013] In a second aspect, in this embodiment, a designated video monitoring and playback device based on an intelligent agent is provided, comprising: A request acquisition module, used to generate an instruction request in response to the user's operation on the front end; A rule generation module, used for generating a tool call link rule according to the instruction request in the live screen real-time stream agent when the instruction request is to call the live screen; The tool calling module is used to call the corresponding tools in sequence to obtain the video stream address of the target monitoring device according to the tool calling link rules and the instruction request, so as to load the corresponding live screen according to the video stream address at the front end.

[0014] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the agent-based designated video surveillance playback method described in the first aspect is implemented.

[0015] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the agent-based designated video monitoring and playback method described in the first aspect is implemented.

[0016] Compared with the related art, the agent-based designated video surveillance playback method, device and storage medium provided in this embodiment generate a command request in response to the user's operation at the front end; when the command request is to call a live screen, in the live screen real-time stream agent, a tool call link rule is generated according to the command request; according to the tool call link rule and the command request, the corresponding tool is called in sequence to obtain the video stream address of the target monitoring device, so as to load the corresponding live screen at the front end according to the video stream address. Through this embodiment, according to the user's command request at the front end, the video stream address of the target monitoring device can be obtained by calling the tool through the corresponding agent, so as to play the live screen of the user-specified monitoring device more conveniently and efficiently, solving the problem of low efficiency in searching for live screens in video surveillance.

[0017] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a hardware structure block diagram of a terminal of a method for playing a designated video surveillance video based on an intelligent agent in an embodiment; Figure 2 is a flow chart of a method for monitoring and playing a designated video based on an agent in one embodiment; Figure 3 is a flow chart of a method for monitoring and playing a designated video based on an intelligent agent in another embodiment; Figure 4 is a schematic diagram of the overall architecture of the front end, back end and intelligent agent platform in one embodiment; Figure 5 is a schematic diagram of the processing flow of an intelligent agent platform in one embodiment; Figure 6 The structure block diagram of a designated video monitoring and playback device based on an intelligent agent in an embodiment.

[0019] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, request acquisition module; 20, rule generation module; 30, tool calling module. DETAILED DESCRIPTION

[0020] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0021] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0022] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal of the agent-based specified video monitoring and playback method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0023] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the agent-based designated video surveillance playback method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0024] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0025] In this embodiment, a method for monitoring and playing a designated video based on an intelligent agent is provided. Figure 2 is a flow chart of the agent-based specified video monitoring and playback method in this embodiment. Figure 2 As shown, the method comprises the following steps: Step S201, generating an instruction request in response to the user's operation on the front end.

[0026] Specifically, the front end refers to the part of the application that the user interacts directly with, which is responsible for presenting the user interface, receiving input, processing instant feedback, and communicating with the back end. The user issues instructions through the front end, and the back end forwards the instruction request to the intelligent agent platform to start the subsequent intelligent processing process. In this process, the back end plays the role of a bridge connecting the front-end user operation and the back-end intelligent processing, ensuring that the instruction request can be accurately and timely delivered to the core link of intelligent processing. In addition, after receiving the instruction request, the back end can also obtain the user account and pass the user account as an extended field to the intelligent agent platform.

[0027] Furthermore, the AI ​​(Artificial Intelligence) assistant function module can be integrated in the front end. Users can issue natural language commands in the form of dialogue on the chat page of the AI ​​assistant in the interface. As a direct window for users to interact with the system, the AI ​​assistant provides users with a more intuitive and convenient operation interface, which greatly simplifies the operation process of users obtaining the live broadcast screen of the device. Among them, users can issue command requests for monitoring live screen calls, monitoring equipment management, and security management according to actual needs, such as "open the live screen of a monitoring device in a certain place" and "check the network status of a monitoring device in a certain place".

[0028] Step S202, when the instruction request is to call a live screen, in the live screen real-time stream agent, a tool calling link rule is generated according to the instruction request.

[0029] Specifically, the agent platform can include agents such as intent recognition agent and live screen real-time stream agent. Each agent is connected to the big model and implements the function of the corresponding agent by calling the interface of the big model. After the agent platform receives the user's instruction request, the user's intention is first identified by the intent recognition agent, and the agent related to the user's intention is determined according to the prompt word description of each agent, so as to forward the user's instruction request to the relevant agent for processing.

[0030] When it is recognized that the instruction request is to call a live broadcast screen, the instruction request may include information such as the name of the monitoring device, the device location, and the device code. The live broadcast screen real-time stream intelligent agent will process the instruction request by utilizing the tool capabilities of the intelligent agent and the reflective ability of the large model, combined with the system prompts preset by the intelligent agent and the information in the instruction request, to intelligently determine the tools required to implement the instruction request and generate tool call chain rules. When the tool chain call rules contain multiple required tools, the tool call order is also included.

[0031] When the agent platform service is started, you can register and define tools in the agent. First, define a tool class and register it, and then define related parameters in the tool class.

[0032] Step S203, according to the tool calling link rules and instruction requests, the corresponding tools are called in sequence to obtain the video stream address of the target monitoring device, so as to load the corresponding live screen at the front end according to the video stream address.

[0033] Specifically, after obtaining the tool calling link rules, the live screen real-time stream agent calls the required tools in sequence according to the tool calling link rules, using the tool names and related parameters in the rules. Tools with different functions obtain the device code and video stream address of the user-specified monitoring device based on the device name, device location, and device code of the monitoring device in the instruction request.

[0034] The intelligent platform will promptly return the real-time stream address of the target monitoring device to the front-end page. After the front-end page recognizes the video stream address returned by the intelligent platform, it will immediately load the live screen in the front-end player, thereby presenting the live screen of the specified monitoring device to the user.

[0035] Through the above steps, the live screen real-time stream intelligent body can generate tool call link rules according to the instruction request for calling the live screen issued by the user at the front end, and perform tool calls according to the tool call link rules to obtain the video stream address of the target monitoring device, so as to play the live screen of the user-specified monitoring device more conveniently and efficiently, and at the same time simplify the user's operation process, solving the problem of inefficient live screen search in video surveillance.

[0036] In some of the embodiments, the step S201 generates an instruction request in response to the user's operation on the front end, including the following steps: In response to the user's text input or voice input, a command request is generated; the command request is a natural language command.

[0037] Specifically, in order to simplify the user's operation process, the front end can provide users with operation methods such as text input and voice input to generate command requests. For example, the characters typed by the user can be received through a text box in the chat page of the AI ​​assistant, or the user's audio stream can be captured through a microphone to obtain input in text form and generate natural language instructions.

[0038] By providing users with text input and language input operation methods in this application, users can perform natural language interaction on the front end and issue command requests more efficiently and conveniently without having to search for designated monitoring devices according to geographical levels, thereby improving the user experience.

[0039] In some embodiments, after generating the instruction request, the method further includes the following steps: The semantic information of the command request is analyzed through the intention recognition agent; based on the semantic information, the command request is forwarded to the relevant agent for processing.

[0040] Specifically, the intelligent agent platform first uses the intent recognition agent to accurately recognize the intent of the user's command request. Specifically, the intent recognition agent is connected to the big model, and the semantic information of the command request is analyzed using the capabilities of the big model. The agent related to the user's intention is determined based on the prompt word description of each agent, and the name of the agent for subsequent processing is returned to forward the user's command request to the relevant agent for processing.

[0041] When it is recognized that the user's intention is related to opening the camera screen, the intelligent agent platform forwards the instruction request to the intelligent agent specifically responsible for real-time stream processing of the live screen for further processing.

[0042] The following is an example of defining an intent recognition agent and a prompt word: When you can answer the user directly, please ignore the helper and reply directly; but when your ability is unable to meet the user's request, please choose one of them to help you answer. The templates are as follows: Call: ... # The name of the selected helper must be selected in [{agent_names}], do not return anything else.

[0043] Reply: ... # The selected helper's reply.

[0044] Among them, agent_names represents the name of the agent (helper) in the agent platform. When the intent recognition agent identifies other agents related to the user's intent, it returns the name of the agent and the agent continues to reply to the user.

[0045] Through the intention recognition agent of the agent platform in this embodiment, it is possible to accurately analyze the semantic information of the user's command request, identify the intention of the command request and perform targeted task allocation, and forward the command request to the intent-related agent for processing.

[0046] In some embodiments, the step S202 of generating a tool call link rule according to the instruction request includes the following steps: The required tools are determined according to the instruction request, and a tool calling link rule including a tool calling sequence is generated accordingly; the tools include a first tool for obtaining a device code of a target monitoring device, and a second tool for obtaining a video stream address.

[0047] Specifically, when the instruction request is to call a live broadcast screen, the instruction request may include information such as the device name, device location, and device code of the monitoring device. In general, the process of obtaining the video stream address of the target monitoring device is: first obtain a device list based on the device name, installation address, and user account, the device list includes information such as the device code, device name, and installation address, and then obtain the corresponding video stream address based on the device code in the device list. In this process, two tools may need to be called, specifically a first tool for obtaining the device code of the target monitoring device, and a second tool for obtaining the video stream address.

[0048] The following is a code example for registering and defining the first tool in a live screen real-time streaming agent.

[0049] First, define a tool class and register it: @register_tool('device_code_tool'); class DeviceCodeTool(BaseTool).

[0050] Secondly, define the relevant parameters in this tool class: description = ('This tool is used to retrieve the device code of the camera device. It supports the device name or the area where it is installed as the query condition. The returned result is a device list, including the device code (device_sn), device name (device_name), installation area (area '), etc.'), params = [ { 'name': 'device_name', 'type': 'string', 'description': 'The name of the camera device', 'required': False, }, { 'name': 'area', 'type': 'string', 'description': 'The location of the camera device', 'required': False, } ], def call(self, params: Union[str, dict], files: List[str]=None, ) ->List[dict].

[0051] The tool description field (discrprition) is the basis for the agent to think about whether the tool is applicable. The parameters (params) are the parameters that the agent needs to pass when calling the tool. The parameter content is obtained through the user's instruction request, and the call function is used to implement the specific logic of the tool.

[0052] When the instruction request includes a device code, a tool call link rule including the second tool is generated; when the instruction request does not include a device code, a tool call link rule including the first tool and the second tool in sequence is generated.

[0053] Among them, if the user already knows the device code of the specified target monitoring device, then a command request containing the device code can be issued. When the command request contains the device code, the video stream address can be directly obtained using the second tool, thereby generating a tool call link rule containing the second tool.

[0054] If the user's instruction request includes the name and / or device location of the specified monitoring device, but does not include the device code, the live screen real-time stream intelligent agent needs to first use the first tool to obtain the device code, and then use the second tool to obtain the video stream address based on the device code, thereby generating a tool call link rule that includes the first tool and the second tool in sequence.

[0055] The following is an example of defining a live streaming agent and prompt words: This agent is designed to help users obtain the device information of the monitoring device, including the video stream address, device location, device name, etc. Before use, please follow the steps below: 1. Extract the device name or device location based on the command request and context entered by the user. If you can extract the device code from the user's input and context, use it directly without using a tool to obtain the device code.

[0056] 2. If the device code is already known, there is no need to use the device_code_tool tool (the first tool). Otherwise, use the device_code_tool tool to obtain the corresponding device code based on the extracted device name or device location.

[0057] 3. Pass the obtained device code as a parameter to the get_device_stream tool (the second tool) to obtain the video stream address of the monitoring device.

[0058] 4. If you cannot obtain the real-time playback stream address of the device, just tell the user directly [Sorry, I cannot obtain the real-time playback stream address of the specified device, please contact the corresponding administrator]. Otherwise, the information you return to the user should follow this format: The real-time video stream address is: http: / / xxx:9007 / xx.

[0059] Please follow these steps carefully to ensure that the task is completed accurately.

[0060] Example: Q: Please help me open the live broadcast screen of xx device; A: OK, we have opened the live broadcast screen of the device for you. The real-time video stream address is: http: / / xxx:9007 / xx.

[0061] By utilizing the tool capabilities of the intelligent agent and the reflective capabilities of the large model in this embodiment, combined with the system prompts preset by the intelligent agent and the information in the instruction request, the tools required to implement the instruction request are intelligently determined and tool call link rules are generated, and then the tool call link rules can be used to perform tool calls in the future to obtain the video stream address.

[0062] In some of the embodiments, in the above step S203, according to the tool calling link rule and the instruction request, calling the corresponding tools in sequence to obtain the video stream address of the target monitoring device includes the following steps: The device code in the command request is input into the second tool to obtain the video stream address of the target monitoring device.

[0063] Specifically, as shown in the above example of live screen real-time stream agent and prompt words, when the instruction request contains a device code, according to the tool call link rules, the device code in the instruction request is input into the second tool for parameter passing to obtain the video stream address of the target monitoring device.

[0064] Input the device name or device location in the command request into the first tool to obtain the device code of the target monitoring device; input the device code into the second tool to obtain the video stream address of the target monitoring device.

[0065] Specifically, as shown in the above example of the live screen real-time stream agent and prompt words, when the instruction request does not contain the device code, according to the tool call link rules, the device name or device location in the instruction request is input into the first tool for parameter passing, and the device code of the target monitoring device is obtained, and then the device code is input into the second tool to obtain the video stream address of the target monitoring device.

[0066] In this embodiment, the tool call is performed according to the tool call link rules combined with the information extracted from the instruction request to obtain the video stream address of the target monitoring device. Through this precise calling method, the live broadcast image of the target monitoring device can be accurately and quickly obtained.

[0067] The present embodiment is described and illustrated below through preferred embodiments.

[0068] Figure 3 is a flow chart of the agent-based specified video monitoring and playback method of this embodiment. Figure 3 As shown, the method comprises the following steps: Step S301, in response to the text input or voice input of the user at the front end, generates a command request; and forwards the command request to the intelligent agent platform through the back end.

[0069] Step S302, analyze the semantic information of the instruction request through the intention recognition agent of the agent platform; according to the semantic information, when the instruction request is to call the live broadcast screen, forward the instruction request to the live broadcast screen real-time stream agent.

[0070] Step S303: determine the required tool according to the instruction request, and generate a tool calling link rule including a tool calling sequence accordingly.

[0071] The tools include a first tool for obtaining a device code of a target monitoring device and a second tool for obtaining a video stream address.

[0072] Step S304, when the instruction request includes a device code, a tool call link rule including the second tool is generated; according to the tool call link rule, the device code in the instruction request is input into the second tool to obtain the video stream address of the target monitoring device.

[0073] Step S305, when the instruction request does not contain a device code, a tool call link rule is generated that includes the first tool and the second tool in sequence; according to the tool call link rule, the device name or device location in the instruction request is input into the first tool to obtain the device code of the target monitoring device; and then the device code is input into the second tool to obtain the video stream address of the target monitoring device.

[0074] Step S306, returning the video stream address of the target monitoring device to the front end, so that the front end can load the corresponding live screen according to the video stream address.

[0075] Figure 4 It is a schematic diagram of the overall architecture of the front-end, back-end and intelligent agent platform in this embodiment. Figure 4As shown in the figure, the front-end (front-end web service) provides a user interaction interface and is responsible for receiving input (such as voice, text) and displaying results (such as chat pages, players). The back-end capability service acts as a bridge between front-end user operations and back-end intelligent processing, and is responsible for forwarding command requests to the intelligent agent platform to start subsequent intelligent processing processes.

[0076] The intelligent agent platform includes intelligent agents with different functions and artificial intelligence (AI) components. The intelligent agents include intent recognition agents, live screen real-time streaming agents, etc. Each intelligent agent is connected to the big model, and the functions of the corresponding intelligent agent are realized by calling the interface of the big model. AI components include device capability components that integrate hardware device functions (such as cameras and microphones), retrieval components that provide data query services, etc., to provide relevant capabilities for the intelligent agents.

[0077] External capability services provide third-party interfaces for the intelligent agent platform, extend intelligent agent functions, and register capability components for managing service registration and discovery.

[0078] Figure 5 is a schematic diagram of the processing flow of the intelligent agent platform in this embodiment. Figure 5 As shown, the backend receives the command request input by the user at the frontend and forwards the command request to the agent platform. The agent platform first recognizes the user's input intent. If the user's intent is related to calling the live screen, the live screen real-time stream agent processes the command request and returns the video stream address of the target monitoring device. The frontend page processes it and plays the live screen in the player. For other user intents other than calling the live screen, other related agents process it and return the processing results.

[0079] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0080] The actual implementation process of this method is as follows: 1. System design stage: Based on the above solution, the system designer carefully formulated unified specifications for the dialogue interface to ensure that the interaction interfaces between the modules in the system are consistent and standardized. At the same time, the system prompt words are uniformly designed to ensure that the prompt information when the user interacts with the system is clear, accurate and consistent with the overall logic. In terms of service operation monitoring design, the designer has a comprehensive plan to ensure that the system operation status can be monitored in real time and early warning can be issued in a timely manner.

[0081] The interaction mode between systems is determined to adopt the microservice HTTP API (web page service interface) protocol mode, which has the advantages of being lightweight, flexible, and easy to expand, and can effectively support efficient communication between various modules of the system. The message service uses high-performance Kafka queue technology, which can achieve reliable message transmission, high-throughput processing, and asynchronous communication under a distributed architecture, greatly improving the performance and stability of the system. The external unified interface uses the httpsapi interface specification through the service gateway to ensure the security of the interface, while supporting streaming returns to meet the needs of real-time data transmission and processing. In addition, the specifications and usage paradigms for accessing large models are clarified to lay the foundation for the smooth access and efficient application of subsequent large models.

[0082] 2. Function realization stage: Development engineers develop related functional modules based on Python and FastAPI frameworks. They strictly follow unified specifications and provide northbound interfaces to ensure seamless connection with other systems or modules. They use the agent framework to successfully access large models, follow the development paradigm of AI components, and accurately implement business logic. Through this development process, the system design is transformed into an actual executable functional module, providing core support for the overall operation of the system.

[0083] 3. Test and verification phase: Test engineers focus on verifying whether devices at different levels and between different users can correctly open the live broadcast screen. This includes simulation tests of various possible scenarios, such as devices at different geographical levels, operations of users with different permissions, etc., to ensure that the system can stably and accurately achieve the function of obtaining live broadcast screens in various actual application scenarios, and to ensure the reliability of the system and user experience.

[0084] 4. Service launch phase: After comprehensive testing and verification, when the system meets the launch standards, the operation and maintenance personnel will officially deploy and launch the system. After the launch, the operation and maintenance personnel will continue to track and observe the system's operating results, including performance indicators, user feedback, etc., to promptly discover and solve possible problems, ensure that the system can provide services to users stably and efficiently, and continuously optimize system performance and user experience.

[0085] In this embodiment, a designated video surveillance playback device based on an intelligent agent is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0086] Figure 6 is a structural block diagram of the agent-based designated video monitoring and playback device of this embodiment, such as Figure 6 As shown, the device comprises: The request acquisition module 10 is used to generate an instruction request in response to the user's operation on the front end; A rule generation module 20 is used to generate a tool call link rule according to the instruction request in the live screen real-time stream agent when the instruction request is to call the live screen; The tool calling module 30 is used to call the corresponding tools in sequence to obtain the video stream address of the target monitoring device according to the tool calling link rules and instruction requests, so as to load the corresponding live screen at the front end according to the video stream address.

[0087] Through the device provided in this embodiment, the live screen real-time stream intelligent body can generate tool call link rules based on the instruction request for calling the live screen issued by the user at the front end, and perform tool calls based on the tool call link rules to obtain the video stream address of the target monitoring device, so as to play the live screen of the user-specified monitoring device more conveniently and efficiently, and at the same time simplify the user's operation process, solving the problem of inefficient live screen search in video surveillance.

[0088] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0089] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0090] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0091] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0092] In addition, in combination with the agent-based designated video surveillance playback method provided in the above embodiments, a storage medium can also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any agent-based designated video surveillance playback method in the above embodiments is implemented.

[0093] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0094] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0095] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0096] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0097] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for monitoring and playing designated videos based on an agent, characterized in that: include: In response to the user's operation on the front end, generating an instruction request; When the instruction request is to call a live screen, in the live screen real-time stream agent, a tool call link rule is generated according to the instruction request; According to the tool calling link rule and the instruction request, the corresponding tools are called in sequence to obtain the video stream address of the target monitoring device, so as to load the corresponding live screen according to the video stream address at the front end.

2. The agent-based designated video monitoring and playback method according to claim 1, characterized in that: After the generating instruction request, the method further comprises: Analyzing semantic information of the command request through an intent recognition agent; According to the semantic information, the instruction request is forwarded to the relevant agent for processing.

3. The agent-based designated video monitoring and playback method according to claim 1, characterized in that: The generating a tool call link rule according to the instruction request includes: Determine the required tool according to the instruction request, and correspondingly generate the tool calling link rule including the tool calling sequence; The tool includes a first tool for obtaining a device code of the target monitoring device, and a second tool for obtaining a video stream address.

4. The agent-based designated video monitoring and playback method according to claim 3, characterized in that: The determining the required tool according to the instruction request and correspondingly generating the tool call link rule including the call sequence includes: When the instruction request includes a device code, generating a tool call link rule including the second tool; When the instruction request does not include a device code, a tool call link rule is generated that includes the first tool and the second tool in sequence.

5. The agent-based designated video monitoring and playback method according to claim 1, characterized in that: The generating of the instruction request in response to the user's operation on the front end includes: In response to the text input or voice input of the user, an instruction request is generated; the instruction request is a natural language instruction.

6. The agent-based designated video monitoring and playback method according to claim 1, characterized in that: The method of calling corresponding tools in sequence to obtain the video stream address of the target monitoring device according to the tool calling link rule and the instruction request includes: The device code in the instruction request is input into the second tool to obtain the video stream address of the target monitoring device.

7. The agent-based designated video monitoring and playback method according to claim 1, characterized in that: The method of calling corresponding tools in sequence to obtain the video stream address of the target monitoring device according to the tool calling link rule and the instruction request includes: Inputting the device name or device location in the instruction request into the first tool to obtain the device code of the target monitoring device; The device code is input into the second tool to obtain the video stream address of the target monitoring device.

8. A designated video monitoring and playback device based on an intelligent agent, characterized in that: include: A request acquisition module, used to generate an instruction request in response to the user's operation on the front end; A rule generation module, used for generating a tool call link rule according to the instruction request in the live screen real-time stream agent when the instruction request is to call the live screen; The tool calling module is used to call the corresponding tools in sequence to obtain the video stream address of the target monitoring device according to the tool calling link rules and the instruction request, so as to load the corresponding live screen according to the video stream address at the front end.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the agent-based designated video monitoring and playback method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the agent-based designated video monitoring and playback method described in any one of claims 1 to 7 are implemented.

Images