Video inspection method and device, server and storage medium

Abstract

The invention provides a video inspection method and device, a server and a storage medium, relates to the technical field of intelligent inspection, and solves the problems of low inspection efficiency and large workload of manual inspection in related technologies. The inspection instruction statement input by the user is analyzed and recognized to dynamically adjust the inspection strategy, intelligent adaptation of different scenes is achieved, automatic execution of the whole process from system login, video data retrieval to report generation is completed, the manpower demand in the operation and maintenance process is reduced, the manpower cost is remarkably reduced, and the inspection efficiency can also be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent inspection, and in particular to a video inspection method and device, a server and a storage medium. BACKGROUND

[0002] In the field of rail transit such as subways and high-speed rails, there are inspection requirements for places such as waiting halls, platforms, equipment rooms and entrances and exits of stations, such as arranging staff to inspect the above-mentioned places and write inspection reports, or setting up monitoring cameras in the above-mentioned places to perform inspection through CCTV (Closed-Circuit Television). In related technologies, the inspection work is completed by manual work, and even if the inspection is performed through CCTV, the staff needs to review the video pictures corresponding to the cameras one by one on the display device to complete the inspection, or check the video pictures displayed in the multiple video windows in the split screen mode one by one, but the details in the video window in the split screen mode are difficult to observe, and frequent zoom-in and zoom-out operations are required. Moreover, for random inspection tasks such as inspection of specific places and specific behaviors, the staff needs to formulate an inspection route and quickly review the video pictures taken by the corresponding monitoring cameras to perform inspection and analysis.

[0003] To this end, in related technologies, the scenes to be inspected still mainly rely on manual inspection, the staff has a large workload and it is difficult to complete the inspection in time, the inspection work is highly repetitive, the manual inspection method is prone to errors and omissions due to the decline in attention of the staff due to long-term repetitive work, and the inspection efficiency is low. SUMMARY

[0004] The present application provides a video inspection method, device, server and storage medium, which solves the problems of low inspection efficiency and large workload of manual inspection in related technologies, and can automatically adjust the inspection task according to real-time scene requirements, realize automatic inspection, reduce the labor input, and improve the inspection efficiency.

[0005] In a first aspect, the present application provides a video inspection method applied to a server, the server being in communication connection with a camera in an inspection place to receive video data taken by the camera, and the video inspection method comprising: based on the received inspection instruction sentence, performing semantic analysis on the inspection instruction sentence to determine an intent label and parameter information of the inspection instruction sentence, the intent label being used to represent a task type corresponding to the inspection instruction sentence, and the parameter information including text of a task content in the inspection instruction sentence; in a case where the intent label is associated with a video inspection type, positioning a target camera and determining an inspection task according to the parameter information, the inspection task being used to analyze and identify a video picture image of the video data; In response to the received user confirmation operation on the inspection task, the tool component is invoked to perform recognition and analysis processing on the video picture image according to the inspection task, and it is determined whether there is a target event associated with the parameter information; In the case where there is a target event, an inspection report is generated, and the video picture image recognized as having the target event is carried in the inspection report.

[0006] In a second aspect, the present application also provides a video inspection device applied to a server, the server being in communication connection with a camera in an inspection place to receive video data shot by the camera, and the video inspection device comprising: An instruction analysis module configured to perform semantic analysis on the received inspection instruction sentence based on the inspection instruction sentence, determine an intent label and parameter information of the inspection instruction sentence, the intent label being used to represent a task type corresponding to the inspection instruction sentence, and the parameter information including text corresponding to task content in the inspection instruction sentence; A task planning module configured to, in the case where the intent label is associated with a video inspection type, locate a target camera and determine an inspection task according to the parameter information, the inspection task being used to perform analysis and recognition on a video picture image of the video data; A task execution module configured to, in response to a received user confirmation operation on the inspection task, invoke the tool component to perform recognition and analysis processing on the video picture image according to the inspection task, and determine whether there is a target event associated with the parameter information; A result generation module configured to, in the case where there is a target event, generate an inspection report, and the video picture image recognized as having the target event is carried in the inspection report.

[0007] In a third aspect, the present application also provides a server, which comprises: One or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the video inspection method of the present application.

[0008] In a fourth aspect, the present application also provides a storage medium storing computer executable instructions, when the computer executable instructions are executed by a processor, the computer executable instructions are used to execute the video inspection method of the present application.

[0009] The present application scheme can integrate a large model with a tool component, analyze and recognize a user input inspection instruction sentence to dynamically adjust an inspection strategy, realize intelligent adaptation in different scenarios, complete full-process automatic execution from login to the system, review of video data to generation of a report, reduce human demand in operation and maintenance process, significantly reduce human cost, and also improve inspection efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 A step schematic diagram of a video inspection method provided by an embodiment of the present application.

[0011] Figure 2 A step schematic diagram of determining an intention label and parameter information provided by an embodiment of the present application.

[0012] Figure 3 A step schematic diagram of executing an inspection task provided by an embodiment of the present application.

[0013] Figure 4 A structure schematic diagram of a video inspection device provided by an embodiment of the present application.

[0014] Figure 5 A structure schematic diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0015] The present application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, and not to limit the embodiments of the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the embodiments of the present application are shown in the drawings, and those skilled in the art should understand that, as long as the technical features are not contradictory, any combination of technical features can constitute an optional embodiment.

[0016] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be exchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually a class, and do not limit the number of objects, for example, the first object can be one or more. In addition, the specification and claims "and / or" means at least one of the connected objects, and the character " / " generally means that the front and rear associated objects are in an "or" relationship. In the description of the present application, "multiple" means two or more, and "several" means one or more.

[0017] In the field of rail transit such as subway and high-speed rail, in order to ensure the safety of operation, the operator usually inspects the station site such as waiting hall, platform, equipment room, entrance and exit and other places. For example, the operator arranges staff to inspect the above-mentioned places and write inspection reports, or sets up monitoring cameras in the above-mentioned places to manage the video data shot by the cameras through CCTV, and inspects through artificial review of the video, such as the staff reviews the video screen corresponding to the camera on the display device one by one to complete the inspection, or reviews the video screen displayed in the multiple video windows displayed in the split screen mode one by one, but the details in the video window in the split screen mode are difficult to observe, and frequent zoom-in and zoom-out operations are required.

[0018] For this reason, the scene to be inspected in the related art still mainly relies on artificial inspection, and the workload of the staff is large and difficult to complete the inspection in time, and the inspection work is highly repetitive, and the artificial inspection method is prone to errors and omissions due to the decline in attention of the staff due to long-term repetitive work, and the inspection efficiency is low.

[0019] Moreover, for some special inspection requirements, such as random assignment of inspection tasks for specific places and specific behaviors, the staff needs to develop an inspection route and quickly review the video screen shot by the corresponding monitoring camera to perform inspection and analysis. Similarly, the artificial inspection method is prone to errors and omissions, and cannot quickly perform inspection.

[0020] For this reason, the present application provides a video inspection method, which is applied to a server in communication connection with a camera in an inspection site to receive video data shot by the camera. Optionally, the server manages the video data shot by the camera through a CCTV system, that is, the video data shot by the camera in the inspection site is uploaded to the server and managed through the CCTV system.

[0021] Figure 1 The step schematic diagram of the video inspection method provided by an embodiment of the present application, in an embodiment, for the inspection requirements of the user, the user can edit the corresponding inspection instruction sentence to send to the server through the terminal device, and the server can respond to the inspection instruction sentence to build the corresponding inspection task to complete the inspection requirements, the specific steps are as follows: Step S110, based on the received inspection instruction sentence, performing semantic analysis on the inspection instruction sentence to determine the intent label and parameter information of the inspection instruction sentence.

[0022] The inspection instruction sentence is a sentence edited by a user in natural language. The user can send the text or voice corresponding to the inspection instruction sentence to the server through a terminal device. The server can determine the corresponding text content through voice recognition after receiving the inspection instruction sentence, for example, for voice. Further, according to the semantic analysis algorithm for processing natural language, for example, in an embodiment, the inspection instruction sentence is analyzed by a rule-based semantic analysis method, that is, the natural language text is processed by using a rule base and pre-set rules. In the rule base, different syntax rules and semantic rules are modeled to enable the computer to understand and interpret the meaning of natural language. Moreover, the text is analyzed as a token sequence, and then rules are used to parse and interpret them. The token sequence is usually generated by using text preprocessing techniques such as tokenization, lemmatization, and part-of-speech tagging. Further, the rule-based processor traverses the token sequence and finds matching rules to understand and extract information. Alternatively, a deep learning-based semantic analysis method can also be used to analyze the semantic analysis of the inspection instruction sentence, that is, a pre-trained large model such as a BERT (Bidirectional Encoder Representations from Transformers) large model, a GPT (Generative Pre-trained Transformer) large model, etc. is carried in the server, and the semantic analysis of the inspection instruction sentence is performed by the large model.

[0023] Further, the intent label and parameter information in the inspection instruction sentence are determined by semantic analysis, wherein the intent label is used to represent the task type corresponding to the inspection instruction sentence, and the parameter information includes the text of the task content in the inspection instruction sentence, that is, the task type and task content specified by the user are determined by the semantic analysis of the inspection instruction sentence, so as to construct the inspection task.

[0024] Step S120, in the case where the intent label is associated with the video inspection type, the target camera is located and the inspection task is determined according to the parameter information.

[0025] The server is pre-configured with multiple intent labels to associate different task types through different labels, such as associating a video inspection type through a label L1, associating a manual inspection type through a label L2, and the like. In a case where it is determined that the currently recognized intent label is associated with the video inspection type, the server needs to locate the target camera and determine the corresponding inspection task, wherein the inspection task is used to analyze and identify the video picture image of the video data. For example, the server determines the target camera and the inspection task according to the parameter information. It is conceivable that the parameter information includes text corresponding to the task content. Based on the text corresponding to the task content in the parameter information, the to-be-inspected target can be determined. For example, the text corresponding to the task content determined by the parameter information includes “A area”, “B area” and “person falling down”. The server can determine that the areas to be inspected include A area and B area based on the parameter information, and then determine the cameras deployed in A area and B area as the target cameras, and further determine the corresponding inspection task as detecting the person falling down in A area and B area.

[0026] In some embodiments, the server can be a large model-based agent that determines the target camera and the inspection task. The large model-based agent is an intelligent system that can learn and make decisions by itself, which is constructed by large-scale pre-training, reinforcement learning, natural language processing, multi-modal technology, etc. In the large model-based agent, reinforcement learning is combined with large-scale pre-training model to further improve the adaptive ability and decision efficiency of the agent, and multi-modal technology is used to fuse different types of data (such as text, image, voice, etc.), so that the agent can process more complex information. Through multi-modal learning, the agent can understand and process multiple forms of input at the same time, providing richer and more accurate feedback.

[0027] Step S130, in response to the received user confirmation operation on the inspection task, the tool component is called to identify and analyze the video picture image according to the inspection task, and it is determined whether there is a target event.

[0028] The server sends the generated inspection task to the terminal device of the user for confirmation. The inspection task is divided into multiple process nodes, and each process node in the inspection task is optionally displayed to the user in a visual manner for the user to browse and confirm the inspection task. After receiving the confirmation operation of the user on the inspection task, in response to the confirmation operation, the server schedules according to the inspection task, calls the tool component to perform identification analysis processing on the video picture image, and determines whether there is a target event, wherein the target event is associated with the parameter information. The corresponding tool component is preset in the server, and the tool component is used to perform related operations in the identification analysis processing process, such as reading data, data analysis, and the like. Through the identification analysis of the video picture image, the server can identify the picture content appearing in the image, and then determine whether there is a target event. For example, the target event to be searched is the situation of personnel falling, and correspondingly, the video picture image obtained from the video data is subjected to person recognition and behavior recognition to determine whether there is a situation of personnel falling in the current image, and then the identification result is determined to determine whether there is a target event.

[0029] In step S140, in the case where the target event exists, an inspection report is generated, and the inspection report carries the video picture image identified as having the target event.

[0030] The server determines that the current identified video picture image has the behavior, action, etc. required to be detected in the task content in the parameter information, and then determines that the image meets the requirements of the inspection instruction sentence, that is, it is determined that the target event exists. In the case where it is determined that the target event exists, the server obtains the current identified video picture image, which is the video picture image having the target event, and the server also generates an inspection report and stores the above obtained video picture image in the inspection report, so that when the user checks the inspection report, the user can check the video picture image identified as having the target event, so as to facilitate the user to confirm and expand subsequent work. Optionally, after the inspection report is generated, the inspection report can be sent to a specified mailbox through SMTP (Simple Mail Transfer Protocol).

[0031] Exemplarily, the server can implement the above-mentioned video inspection method by constructing a corresponding architecture, and the server includes a user interaction layer, an intelligent planning layer, and a tool execution layer. The user interaction layer is used for interaction with the user, such as providing a natural language interaction interface, and optionally, a web page is used to display to the user for the user to input an inspection instruction sentence thereon, and the user interaction layer can also be used to analyze the inspection instruction sentence input by the user to determine an intent label and parameter information. The intelligent planning layer is used to determine an inspection task and implement inspection of a place through task planning, such as being configured with a dynamic planning engine and a multi-modal large language model to analyze and identify the received intent label and parameter information to plan a corresponding task. The tool execution layer is used to call a pre-configured tool component to execute the determined inspection task.

[0032] As can be known from the above scheme, the scheme can integrate a large model and a tool component, analyze and identify an inspection instruction sentence input by a user to dynamically adjust an inspection strategy, realize intelligent adaptation of different scenes, complete full-process automatic execution from login to a system, review of video data, to generation of a report, reduce human demand in an operation and maintenance process, significantly reduce human cost, and also improve inspection efficiency.

[0033] In an embodiment, Figure 2 A step schematic diagram for determining an intent label and parameter information provided by an embodiment of the present application is shown in the figure. When performing semantic analysis on an inspection instruction sentence, an intent label and parameter information can be determined through a multi-round polling mechanism. The specific steps include steps S210-S240: Step S210, in a case where it is determined that the inspection instruction sentence has an exception, outputting a first prompt sentence.

[0034] Step S220, analyzing a reply text received in response to the first prompt sentence to determine whether a first label and a first parameter conform to the inspection instruction sentence.

[0035] Step S230, in a case where the first label does not conform to the inspection instruction sentence, correcting the first label, and taking the corrected first label as an intent label of the inspection instruction sentence.

[0036] Step S240, in a case where the first parameter does not conform to the inspection instruction sentence, correcting the first parameter, and taking the corrected first parameter as parameter information of the inspection instruction sentence.

[0037] The inspection instruction sentence is analyzed to determine whether it is abnormal. Optionally, for unclear, instruction conflict, etc. in the inspection instruction sentence, so that the server cannot determine the target camera and cannot determine the task content to be executed, it can be determined that the current inspection instruction sentence is an abnormal sentence. For example, the user inputs the inspection instruction sentence as "check the shooting content of the camera on the platform", the server can determine that the inspection instruction sentence is abnormal after analysis, and outputs a first prompt sentence for confirming whether the first label and the first parameter in the inspection instruction sentence, wherein the first label is an intent label obtained by initial identification of the inspection instruction sentence, and the first parameter is text in the parameter information. The first prompt sentence carries the first label and the first parameter, and the server communicates with the terminal device of the user to make the terminal device of the user display the first prompt sentence to the user for confirmation. For the above example, the first label carried in the first prompt sentence generated according to the inspection instruction sentence can be "video inspection type", and the first parameter can be "all platforms" or "specific platforms", for example, the displayed first prompt sentence is "the current task is a video inspection type task, please confirm whether to check all platforms".

[0038] It is conceivable that for the reply text fed back by the user through the terminal device, the server parses the reply text to determine whether the first label and the first parameter conform to the inspection instruction sentence. In this case, the first prompt sentence is fed back to confirm whether the task content obtained by the current analysis conforms to the user's intention. For example, when the user's feedback reply text can determine that the analyzed task content conforms to the user's intention, the server performs corresponding task planning operation, and then determines the inspection task. In the case where the first label or the first parameter in the first prompt sentence does not conform to the inspection instruction sentence, the first label and / or the first parameter are corrected, such as the first label does not conform to the inspection instruction sentence, the first label is corrected, and the first parameter does not conform to the inspection instruction sentence, the first parameter is corrected, so that the modified first label is used as the intent label and the modified first parameter is used as the parameter information. In an embodiment, the server can determine the correction information of the first label or the first parameter carried in the reply text by parsing the reply text. For example, the reply text corresponding to the above first prompt sentence is "check the camera on platform A and platform B", the server can determine that the first parameter does not conform to the inspection instruction sentence by parsing the reply text, and the first parameter is modified to "specific platform (platform A and platform B)". Optionally, after correcting the first label and / or the first parameter, a second prompt sentence can be generated for the user to confirm, wherein the second prompt sentence carries a second label and a second parameter, the second label is the modified first label, and the second parameter is the modified second parameter.

[0039] To this end, the intention label and parameter information are determined by using a multi-round polling mechanism, and the scheme can accurately determine the task configured by the user, and further better plan the inspection task.

[0040] In some embodiments, after determining the inspection task, the server can also visualize the inspection task to show the user. Optionally, the server determines all process nodes in the inspection task and visualizes each process node to generate a visualization chart. The visualization chart is a chart that can be used for display. In the process of visualizing the process nodes, the server can generate text and / or symbols associated with the process nodes by analyzing the process nodes, so as to facilitate visualization. Then, the visualization chart is sent to the terminal device of the user to allow the user to confirm the inspection task, that is, the terminal device of the user displays the visualization chart on the display interface thereof after receiving the visualization chart, so as to allow the user to view the inspection task planned by the server.

[0041] Optionally, the user can adjust the visualization chart, such as adjusting the execution order of the process nodes, modifying the task content corresponding to the process nodes, and feeding back to the server through the terminal device. The server determines that there is a process node that does not meet the inspection instruction sentence in the current inspection task in response to the received adjustment operation of the user on the visualization chart, and further generates a new process node. For example, all process nodes in the inspection task or a single generated process node are regenerated to revise the incorrect process node. To this end, the server re-visualizes each process node based on the regenerated process node to generate a new visualization chart and send it to the terminal device of the user for the user to confirm again. To this end, after generating the inspection task, the multi-round polling mechanism is used to continuously communicate data with the terminal device to allow the user to confirm whether the inspection task currently planned by the server meets the requirements, and further allows the server to continuously correct the planned inspection task, thereby providing a better solution for the user.

[0042] In some embodiments, the tool component is used to perform an action to complete the execution of the inspection task when invoked. The tool component includes a first type of tool and a second type of tool, wherein the first type of tool is used to read video data, and the second type of tool is used to identify a person in an image. Figure 3 The steps of performing the inspection task provided by an embodiment of the present application are shown in the flowchart. In an embodiment, the server responds to the confirmation operation of the user to invoke the tool component to analyze and identify the video picture image in the video data, and the specific steps are as follows: Step S310, determining a first process node and a second process node in the inspection task.

[0043] In a case where the first flow node is executed, the video data corresponding to the target camera is read by calling the first type of tool.

[0044] In a case where the second flow node is executed, the video picture image obtained from the video data corresponding to the target camera is subjected to person recognition and person behavior analysis by calling the second type of tool, to determine whether a target event exists.

[0045] The first flow node is a node for instructing reading of the video data corresponding to the target camera, and the second flow node is a node for instructing analysis and recognition of the video data. Optionally, for division of different flow nodes, a corresponding flag bit can be set to distinguish when the inspection task is generated, that is, the flag bit is used to distinguish different flow nodes, so that the first flow node and the second flow node in the inspection task can be determined based on the flag bit.

[0046] When the first flow node is executed, after the target camera is located, the video data corresponding to the target camera is read by calling the first type of tool, so as to obtain the video data corresponding to the target camera. It can be conceived that the target camera is a camera arranged in a place to be inspected determined in the parameter information. In this regard, the server can obtain the video picture image obtained from the video data corresponding to the target camera by calling the first type of tool.

[0047] Optionally, in a case where the video data is managed by the CCTV system in the server, the first type of tool includes a Playwright tool, a password management tool, and a CMDB query tool. The Playwright tool is an automatic test tool that can run across platforms (Windows, macOS, Linux) and has powerful browser automation capabilities, and can be applied to test, monitoring, and other scenarios. The password management tool is used to record account and password and can automatically fill in the account and password when called. The CMDB query tool supports creating, updating, and deleting various configuration items, such as connected devices and configured software, and can generate a corresponding list for auditing and analysis. In this regard, the video picture image can be obtained by calling the first type of tool, such as loading the login page of the CCTV system by the Playwright tool, and then automatically filling in the account and password by the password management tool to log in to the CCTV system, and obtaining the camera list by the CMDB (Config Management Database, configuration management database) query tool, the camera list recording information of all cameras arranged in the inspection place, such as device identifier, device location, and data storage location, and further finding the target camera by the Playwright tool and obtaining the video picture image of the video data of the target camera.

[0048] In the execution of the second flow node, the video picture image is subjected to person recognition and person behavior analysis by calling the second type of tool to determine whether a target event exists. The second type of tool is used for analysis and recognition of the video picture image. Optionally, the second type of tool includes a behavior recognition tool that analyzes and recognizes the person in the video picture image by calling a YOLO (You Only Look Once, object detection) model, which is a neural network model for recognizing and behavior analyzing a target object. For example, the YOLO model can be a model obtained by pre-training on a data set including various person actions. The server can call the YOLO model through the behavior recognition tool to recognize the person in the video picture image and determine the action of the recognized person. In the case where the action of the recognized person is a target action, it is determined that a target event exists. The target action is associated with the task content. It is conceivable that the task content in the parameter information records the content of recognizing a person falling down, so that falling down is taken as the target action, and then when the person action is recognized as falling down, it is determined that a target event exists at present. A patrol report is generated to show the user the recognition result of the currently executed patrol task.

[0049] Therefore, the present scheme integrates large models with tool components, calls the tool components to execute patrol tasks, stably operates in a complex CCTV environment, and realizes the whole process of unattended operation from logging into the system, reviewing videos to generating reports. Compared with the manual patrol mode, the present scheme can reduce the time consumption of patrol, eliminate the error of manual operation, and effectively improve the patrol efficiency.

[0050] Figure 4 The structure diagram of the video patrol device provided by an embodiment of the present application is shown in the figure. The device is used to execute the video patrol method provided by the above-mentioned embodiment and has the corresponding functional modules and beneficial effects of the execution method. As shown in the figure, the device includes an instruction analysis module 401, a task planning module 402, a task execution module 403, and a result generation module 404. The device is applied to a server that is in communication connection with a camera in a patrol place to receive video data shot by the camera.

[0051] The instruction analysis module 401 is configured to perform semantic analysis on the received patrol instruction sentence, determine the intent label and parameter information of the patrol instruction sentence, and the intent label is used to represent the task type corresponding to the patrol instruction sentence, and the parameter information includes the text of the task content in the patrol instruction sentence. The task planning module 402 is configured to, in the case where the intent label is associated with the video patrol type, locate the target camera and determine the patrol task according to the parameter information, and the patrol task is used for analysis and recognition of the video picture image of the video data. The task execution module 403 is configured to, in response to the received confirmation operation of the user on the inspection task, call the tool component to perform identification and analysis processing on the video picture image according to the inspection task, and determine whether there is a target event associated with the parameter information; The result generation module 404 is configured to, in the case where there is a target event, generate an inspection report carrying the video picture image identified as having a target event.

[0052] On the basis of the above embodiment, the instruction analysis module 401 is specifically configured to: In the case where it is determined that the inspection instruction statement has an exception, output a first prompt statement, the first prompt statement being used to confirm with the user whether the first label and the first parameter conform to the inspection instruction statement, the first label being an intent label obtained by initial identification of the inspection instruction statement, and the first parameter being a text in the parameter information; Parse the received reply text corresponding to the first prompt statement to determine whether the first label and the first parameter conform to the inspection instruction statement; In the case where the first label does not conform to the inspection instruction statement, modify the first label, and take the modified first label as the intent label of the inspection instruction statement; In the case where the first parameter does not conform to the inspection instruction statement, modify the first parameter, and take the modified first parameter as the parameter information of the inspection instruction statement.

[0053] On the basis of the above embodiment, the device further comprises a visualization module, which is configured to: Determine all process nodes in the inspection task, and perform visualization processing on each process node to generate a visualization chart; Send the visualization chart to the terminal device of the user to enable the user to confirm the inspection task.

[0054] On the basis of the above embodiment, the visualization module is further configured to: In response to the received adjustment operation of the user on the visualization chart, determine that there is a process node in the inspection task that does not satisfy the inspection instruction statement, and regenerate a new process node; Based on the regenerated process node, perform visualization processing on each process node again to generate a new visualization chart and send the new visualization chart to the terminal device of the user.

[0055] On the basis of the above-mentioned embodiments, the tool set comprises first-type tools and second-type tools, the first-type tools are used for reading video data, and the second-type tools are used for identifying persons in images. The task execution module 403 is specifically configured to: determine a first flow node and a second flow node in the inspection task, the first flow node is a node used for instructing reading of video data corresponding to the target camera, and the second flow node is a node used for instructing analysis and identification of the video data; in a case of executing the first flow node, reading the video data corresponding to the target camera by calling the first-type tools; in a case of executing the second flow node, performing person identification and person behavior analysis on a video picture image obtained from the video data corresponding to the target camera by calling the second-type tools, to determine whether the target event exists.

[0056] On the basis of the above-mentioned embodiments, in a case of managing video data in the server through a CCTV system, the first-type tools comprise a Playwright tool, a password management tool and a CMDB query tool. The task execution module 403 is further configured to: call a headless browser to load a login page of the CCTV system through the Playwright tool; automatically fill in an account and a password to log in to the CCTV system through the password management tool; obtain a camera list through the CMDB query tool, and find the target camera and obtain a video picture image of video data of the target camera through the Playwright tool.

[0057] On the basis of the above-mentioned embodiments, the second-type tools comprise a behavior identification tool calling a YOLO model, and the YOLO model is a neural network model used for identifying and behavior analyzing a target object. The task execution module 403 is further configured to: identify a person in a video picture image and determine a motion of the identified person by calling the YOLO model through the behavior identification tool; in a case that the motion of the identified person is a target motion, determine that the target event exists, the target motion is associated with the task content.

[0058] It is worth noting that, in the above-mentioned embodiments of the device, each module is only divided according to functional logic, but is not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of each module are only for convenient mutual distinction, and do not limit the protection scope of the embodiments of the present application.

[0059] Figure 5 A structural diagram of a server provided by an embodiment of the present application is shown in the figure. The device is used to execute the video inspection method provided by the above embodiment and has the corresponding function modules and beneficial effects of the execution method. As shown in the figure, the device includes a processor 501, a memory 502, an input device 503, and an output device 504. The number of the processor 501 can be one or more, and one processor 501 is taken as an example in the figure. The processor 501, the memory 502, the input device 503, and the output device 504 can be connected through a bus or other means, and connection through a bus is taken as an example in the figure. The memory 502 is a kind of computer readable storage medium, which can be used to store software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the video inspection method in the embodiments of the present application. The processor 501 executes the software programs, instructions, and modules stored in the memory 502, thereby executing corresponding various function applications and data processing, that is, implementing the above-mentioned video inspection method.

[0060] The memory 502 can mainly include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required by a function. The data storage area can store data recorded or created during use, etc. In addition, the memory 502 can include a high-speed random access memory and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 502 can further include a memory remotely arranged with respect to the processor 501, which can be connected to the device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0061] The input device 503 can be used to input corresponding digital or character information to the processor 501 and generate key signal inputs related to user settings and function control of the device. The output device 504 can be used to send or display key signal outputs related to user settings and function control of the device.

[0062] The embodiments of the present application provide a storage medium having computer executable instructions, which, when executed by a processor, are used to perform the related operations in the video inspection method provided by any of the embodiments of the present application.

[0063] Computer-readable storage media includes permanent and non-permanent, removable and non-removable media implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0064] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0065] Note that the above are only the preferred embodiments of the present application and the principles of technology used. Those skilled in the art will understand that the present application is not limited to the specific embodiments herein, and that various obvious changes, re-adjustments and substitutions can be made without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A video inspection method, characterized in that, The method is applied to a server, which is communicatively connected to cameras within the inspection area to receive video data captured by the cameras. The video inspection method includes: Based on the received inspection instruction statement, semantic analysis is performed on the inspection instruction statement to determine the intent tag and parameter information of the inspection instruction statement. The intent tag is used to represent the task type corresponding to the inspection instruction statement, and the parameter information includes the text of the corresponding task content in the inspection instruction statement. When the intent tag is associated with the video inspection type, the target camera is located and the inspection task is determined according to the parameter information. The inspection task is used to analyze and identify the video images of the video data. In response to the user's confirmation of the inspection task, the tool component is invoked to identify and analyze the video image according to the inspection task, and to determine whether a target event exists, the target event being associated with the parameter information; If the target event exists, an inspection report is generated, which includes video images that identify the presence of the target event.

2. The video inspection method according to claim 1, characterized in that, The semantic analysis of the inspection command statement to determine its intent label and parameter information includes: If it is determined that the inspection instruction statement is abnormal, a first prompt statement is output. The first prompt statement is used to confirm with the user whether the first label and the first parameter are consistent with the inspection instruction statement. The first label is the intent label obtained by the initial recognition of the inspection instruction statement, and the first parameter is the text in the parameter information. Parse the received response text corresponding to the first prompt statement to determine whether the first tag and the first parameter conform to the inspection instruction statement; If the first tag does not conform to the inspection instruction statement, the first tag is corrected and the corrected first tag is used as the intent tag of the inspection instruction statement. If the first parameter does not conform to the inspection instruction statement, the first parameter is corrected, and the corrected first parameter is used as the parameter information of the inspection instruction statement.

3. The video inspection method according to claim 1, characterized in that, After locating the target camera and determining the inspection task based on the parameter information, the method further includes: Identify all process nodes in the inspection task and visualize each process node to generate a visual chart; The visualization chart is sent to the user's terminal device so that the user can confirm the inspection task.

4. The video inspection method according to claim 3, characterized in that, The method further includes: In response to the received user adjustment operation on the visualization chart, it is determined that there are process nodes in the inspection task that do not meet the inspection instruction statement, and new process nodes are regenerated. Based on the regenerated process nodes, each process node is re-visualized to generate new visualization charts, which are then sent to the user's terminal device.

5. The video inspection method according to claim 1, characterized in that, The tool components include a first type of tool and a second type of tool. The first type of tool is used to read video data, and the second type of tool is used to identify people in the image. In response to the received user confirmation of the inspection task, the system invokes a tool component to perform recognition and analysis on the video image according to the inspection task, and determines whether a target event exists, including: The first process node and the second process node in the inspection task are determined. The first process node is used to instruct the reading of video data corresponding to the target camera, and the second process node is used to instruct the analysis and identification of the video data. When executing the first process node, the video data corresponding to the target camera is read by calling the first type of tool; When executing the second process node, the second type of tool is invoked to perform person recognition and person behavior analysis on the video images obtained from the video data corresponding to the target camera, in order to determine whether the target event exists.

6. The video inspection method according to claim 5, characterized in that, When video data is managed via a CCTV system on the server, the first type of tools includes Playwright tools, password management tools, and CMDB query tools. When executing the first process node, reading the video data corresponding to the target camera by calling a first type of tool includes: The Playwright tool is used to load the CCTV system's login page using a headless browser. The password management tool automatically fills in the username and password to log in to the CCTV system; The CMDB query tool is used to obtain a list of cameras, and the Playwright tool is used to locate the target camera and obtain the video image of the target camera's video data.

7. The video inspection method according to claim 5 or 6, characterized in that, The second type of tools includes behavior recognition tools that call the YOLO model, which is a neural network model used for identifying and analyzing the behavior of target objects; When executing the second process node, the process involves using a second type of tool to perform person recognition and behavior analysis on the video images obtained from the video data corresponding to the target camera, in order to determine whether the target event exists. This includes: The behavior recognition tool uses the YOLO model to identify people in video images and determine the actions of the identified people. If the identified person's action is the target action, then the existence of the target event is determined, and the target action is associated with the task content.

8. A video inspection device, characterized in that, The video inspection device is applied to a server, which is communicatively connected to cameras within the inspection area to receive video data captured by the cameras. The instruction analysis module is configured to perform semantic analysis on the received inspection instruction statement to determine the intent tag and parameter information of the inspection instruction statement. The intent tag is used to indicate the task type corresponding to the inspection instruction statement, and the parameter information includes the text of the corresponding task content in the inspection instruction statement. The task planning module is configured to locate the target camera and determine the inspection task based on the parameter information when the intent tag is associated with the video inspection type. The inspection task is used to analyze and identify the video images of the video data. The task execution module is configured to respond to the user's confirmation operation for the inspection task, and according to the inspection task, call the tool component to perform recognition and analysis processing on the video image, and determine whether there is a target event, the target event being associated with the parameter information; The result generation module is configured to generate an inspection report when the target event exists, and the inspection report carries video images that are identified as showing the presence of the target event.

9. A server, characterized in that, The server includes: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the video inspection method as described in any one of claims 1-7.

10. A storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a processor, are used to perform the video inspection method as described in any one of claims 1-7.

Images