Intelligent monitoring method and device based on human-computer interaction

Abstract

The invention belongs to the technical field of video monitoring, and particularly relates to an intelligent monitoring method and device based on human-computer interaction. The method comprises the following steps: generating a monitoring area according to a click operation of a user on a monitoring picture shot by a first camera device in a joint defense camera and / or a given language prompt text; performing pixel feature point matching on an area with the same semantic identifier as the monitoring area in a monitoring picture shot by a second camera device in the joint defense camera, and marking the matched area as an extended monitoring area; generating a monitoring code according to a reference video which is uploaded by a user and carries a monitoring event and a description text of the monitoring event; and performing video stream monitoring on the monitoring area and the extended monitoring area, and comparing each frame code in the video stream with the monitoring code to determine whether the monitoring event occurs. According to the invention, the setting precision of the monitoring area is improved, and the abnormal event can be monitored more accurately.

Description

Technical Field

[0001] The present application belongs to the field of video surveillance technology, and in particular relates to an intelligent surveillance method and device based on human-computer interaction. Background Art

[0002] Smart surveillance systems on the market now have the functions of setting monitoring areas and pushing special event recordings and messages, helping users to pay attention to special events that occur in certain important areas in a timely manner, such as a pet cat drinking water in the toilet. However, these two functions have the following shortcomings.

[0003] The nursing scene requires users to circle and set the image range displayed on the screen with their fingers. The existing method is not suitable for application scenarios with a relatively precise monitoring area and complex background. Because it is limited by the resolution of the screen display image, it takes a lot of time for users to circle a complex and precise monitoring area.

[0004] The existing solution cannot reasonably expand the protection area of ​​shaking head type cameras or multi-channel camera joint defense scenarios, and requires users to perform multiple tedious settings of cameras at different angles or different positions in the same large monitoring area.

[0005] The definition of special events is limited and vague. The existing technology generally defines special events with only a few preset special event labels such as abnormal screen changes, the appearance of specific people or animals, and they cannot be expanded. In particular, the definition of abnormal human behavior in the surveillance screen is rigid and vague, and it is impossible to judge fine abnormal behavior. Summary of the invention

[0006] In order to solve the above technical problems, the present application provides an intelligent monitoring method and device based on human-computer interaction.

[0007] The first aspect of the present application provides an intelligent monitoring method based on human-computer interaction, the method comprising:

[0008] Generate a monitoring area according to a click operation of acquiring a monitoring picture taken by a first camera device and / or a language prompt text given by the first camera device;

[0009] Perform pixel feature point matching on an area in the monitoring picture taken by the second camera device that has the same semantic identifier as the monitoring area, and mark the matched area as an extended monitoring area;

[0010] Obtaining a monitoring code generated by a reference video and a description text of a monitoring event, wherein the reference video includes the monitoring event;

[0011] Video stream monitoring is performed on the monitoring area and the extended monitoring area, and each frame code in the video stream is compared with the monitoring code to determine whether the monitoring event occurs.

[0012] Preferably, generating a monitoring area according to a click operation of acquiring a monitoring screen captured by the first camera device and / or a language prompt text given by the first camera device includes:

[0013] Encoding the monitoring picture taken by the first camera device to generate an image coding vector;

[0014] Performing image segmentation on the monitoring screen captured by the first camera device, and semantically marking each segmented area;

[0015] Encoding pixels in each area of ​​the semantic mark to generate a semantic encoding vector;

[0016] Superimposing the image coding vector and the semantic coding vector to form a superimposed vector;

[0017] Encoding the coordinates on the monitoring screen captured by the first camera device and / or the language prompt text to generate a coding vector to be matched;

[0018] A similarity operation is performed between the encoding vector to be matched and the superposition vector to obtain a monitoring area.

[0019] Preferably, obtaining a monitoring code generated by a reference video and a description text of a monitoring event includes:

[0020] Collectively encoding each frame in the reference video and the description text to form a set of prompt codes;

[0021] The prompt codes corresponding to each frame in the reference video are counted to generate the monitoring code.

[0022] Preferably, comparing each frame code in the video stream with the monitoring code to determine whether the monitoring event occurs includes:

[0023] When the similarity between any frame code in the video stream and the monitoring code exceeds a set value, the frame in the video stream is used as a key frame;

[0024] When the length of the continuous frame sequence formed by the key frames exceeds a threshold, an abnormal event alarm behavior is triggered.

[0025] The second aspect of the present application provides an intelligent monitoring device based on human-computer interaction, the device comprising:

[0026] A monitoring area generating module, used for generating a monitoring area according to a click operation of acquiring the monitoring picture taken by the first camera device, and / or a given language prompt text;

[0027] An extended monitoring area generation module is used to match pixel feature points of an area in the monitoring picture taken by the second camera device that has the same semantic identifier as the monitoring area, and mark the matched area as the extended monitoring area;

[0028] A monitoring code generation module, used to generate a monitoring code from a reference video and a description text of a monitoring event, wherein the reference video includes the monitoring event;

[0029] The video monitoring module is used to monitor the video stream of the monitoring area and the extended monitoring area, compare each frame code in the video stream with the monitoring code, and determine whether the monitoring event occurs.

[0030] Preferably, the monitoring area generation module includes:

[0031] An image encoding unit, used to encode the monitoring picture taken by the first camera device to generate an image encoding vector;

[0032] A semantic identification unit, used to perform image segmentation on the monitoring screen captured by the first camera device, and to perform semantic identification on each segmented area;

[0033] A semantic coding unit, used to encode pixels in each area of ​​the semantic marker to generate a semantic coding vector;

[0034] A superposition vector generating unit, used for superimposing the image coding vector and the semantic coding vector to form a superposition vector;

[0035] A unit for generating a coding vector to be matched, used for encoding the coordinates on the monitoring screen captured by the first camera device and / or the language prompt text to generate a coding vector to be matched;

[0036] The similarity calculation unit is used to perform a similarity calculation on the encoding vector to be matched and the superposition vector to obtain a monitoring area.

[0037] Preferably, the monitoring code generation module includes:

[0038] A single-frame prompt code generation unit, used for collectively encoding each frame in the reference video and the description text to form a set of prompt codes;

[0039] The video monitoring code generation unit is used to collect statistics on the prompt codes corresponding to each frame in the reference video to generate the monitoring code.

[0040] Preferably, the video monitoring module includes:

[0041] A key frame determination unit, configured to use any frame in the video stream as a key frame when the similarity between the frame code in the video stream and the monitoring code exceeds a set value;

[0042] The alarm behavior generating unit is used to trigger an abnormal event alarm behavior when the length of the continuous frame sequence formed by the key frames exceeds a threshold.

[0043] The third aspect of the present application provides a computer device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent monitoring method based on human-computer interaction as described above.

[0044] A fourth aspect of the present application provides a readable storage medium, which stores a computer program. When the computer program is executed by a processor, it is used to implement the intelligent monitoring method based on human-computer interaction as described above.

[0045] This application improves the accuracy of monitoring area settings and can monitor abnormal events more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of a preferred embodiment of the intelligent monitoring method based on human-computer interaction of the present application.

[0047] Figure 2 This application Figure 1 A schematic diagram of monitoring area settings within a monitoring screen of the illustrated embodiment.

[0048] Figure 3 This application Figure 1 A flow chart of monitoring area generation in the illustrated embodiment.

[0049] Figure 4 This application Figure 1 A schematic diagram of generating an extended monitoring area within a monitoring picture captured by a second camera device in the joint defense camera in the illustrated embodiment.

[0050] Figure 5 It is a computer system architecture diagram for implementing the intelligent monitoring method based on human-computer interaction of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the implementation of this application clearer, the technical scheme in the implementation of this application will be described in more detail in combination with the drawings in the implementation of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described implementation is a part of the implementation of this application, not all of the implementations. The implementation described below with reference to the drawings is exemplary and is intended to be used to explain this application, and cannot be understood as a limitation on this application. Based on the implementation in this application, all other implementations obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The implementation of this application is described in detail below in combination with the drawings.

[0052] The first aspect of the present application provides an intelligent monitoring method based on human-computer interaction, which is used in a monitoring camera device, such as Figure 1 As shown, the method mainly includes:

[0053] Step S100: generating a monitoring area according to a click operation of obtaining a monitoring screen captured by a first camera device of a user and / or a given language prompt text;

[0054] Step S200: performing pixel feature point matching on an area in the monitoring image captured by the second camera device that has the same semantic identifier as the monitoring area, and marking the matched area as an extended monitoring area;

[0055] Step S300, obtaining a monitoring code generated by a reference video and a description text of a monitoring event, wherein the reference video includes the monitoring event;

[0056] Step S400: Perform video stream monitoring on the monitoring area and the extended monitoring area, compare each frame code in the video stream with the monitoring code, and determine whether the monitoring event occurs.

[0057] The present application specifies the monitoring area of ​​a single camera in step S100. The user only needs to click on the approximate location of the target monitoring area in the image or describe the care scene in text. The interactive system will automatically lock the monitoring area, or output a small number of accurate monitoring area guesses for the user to click and select, greatly simplifying the interactive process of setting accurate monitoring areas.

[0058] The first camera device and the second camera device of the present application are combined to form a joint defense camera device. The joint defense camera device can also include multiple ones, and any one of them can be selected as the first camera device. The other camera devices can be used as the second camera device in turn to participate in the joint data processing with the first camera device of the present application.

[0059] like Figure 2As shown in the figure, there is a "monitoring area description input box" at the bottom of the monitoring screen. After the user enters the language prompt text, the system automatically matches the area segmented in advance by the zero-sample learning image segmentation model of the arbitrary object semantic segmentation model. Figure 2 The area where instance 1-1 is located, the area where instance 4 is located, and the area where instance 5 is located, and then the user can select the desired area as the monitoring area. In addition, the user can also directly click on the monitoring screen, and the area including the clicked screen coordinates will be displayed to the user. The area here is also the area pre-divided by the model. After the user confirms, the area can be used as the monitoring area. In an alternative implementation, the above two methods can also be combined to accurately and quickly determine the monitoring area.

[0060] In some optional implementations, generating a monitoring area according to a user's click operation on a monitoring image captured by a first camera device in a joint defense camera, and / or a given language prompt text, includes:

[0061] Step S101, encoding the monitoring picture taken by the first camera device to generate an image coding vector;

[0062] Step S102: segmenting the surveillance image captured by the first camera device, and semantically marking each segmented area;

[0063] Step S103, encoding pixels in each area of ​​the semantic marker to generate a semantic encoding vector;

[0064] Step S104, superimposing the image coding vector and the semantic coding vector to form a superimposed vector;

[0065] Step S105: Encode the coordinates of the monitoring screen captured by the first camera device clicked by the user, and / or the language prompt text, to generate a coding vector to be matched;

[0066] Step S106: perform a similarity operation on the encoding vector to be matched and the superposition vector to obtain a monitoring area.

[0067] refer to Figure 3In this embodiment, in step S101, the current monitoring screen is first encoded into an image coding vector by an image encoder. At the same time, in step S102, the monitoring screen taken by the first camera device can be segmented based on the semantic segmentation model. The semantic encoder will regard the current monitoring area as a semantic identification pixel and perform semantic encoding on it. Then, in steps S103-S104, the semantic coding vector will be superimposed with the image coding vector. Then, in step S106, it is input to the semantic identification image decoder. The semantic identification image decoder will calculate the similarity between the to-be-matched coding vector and the superimposed vector according to the coordinates of the screen position prompt point and the language prompt information given by the user in step S105, through the to-be-matched coding vector encoded by the prompt encoder, so as to solve the monitoring area. Users only need to perform such a small number of interactive operations to get their expected monitoring area.

[0068] In order to cope with the scenario of expanding the protection area of ​​multi-channel camera joint defense and shaking head intelligent monitoring, the present application proposes in step S200 to perform instance matching on the instance segmentation results of multiple pictures to expand the monitoring area desired by the user.

[0069] like Figure 4 As shown, assuming that the user Figure 2 In the intelligent monitoring screen shown, select Instance 1-1 as the monitoring area, and the system will display the same prompt in Figure 4 In the monitoring picture shown in the figure, the regions are matched to obtain instances 1-2, 2, and 3. Since the semantic identifiers of instances 1-1 and 1-2 are the same (represented by rectangles here), and by matching the feature points of all pixels in the same semantic identifier, a large number of semantic pixels are found to be similar, so instance 1-2 can be considered as an extension of instance 1-1, and is automatically marked as an extended monitoring area without any other operations by the user.

[0070] This application combines image instance segmentation with image feature point matching to achieve matching and expansion of monitoring areas for images taken at different monitoring orientations, thereby reducing the complexity of user operations for setting monitoring areas for multi-screen joint defense.

[0071] Then, in step S300, the application uses an intelligent customer service robot to guide users to expand definitions and more accurately describe special events. According to the reference video uploaded by the user with the monitored event and the description text of the monitored event, the event to be monitored is coded and described, so that when monitoring the video stream, the encoding of the video frame can be compared with the above encoding that can characterize the abnormal event, thereby achieving accurate description and monitoring of the abnormal event.

[0072] In some optional implementations, obtaining a monitoring code generated from a reference video and a description text of a monitoring event includes:

[0073] Step S301, collectively encoding each frame in the reference video and the description text to form a set of prompt codes;

[0074] Step S302: Count the prompt codes corresponding to each frame in the reference video to generate the monitoring code.

[0075] In step S301, each frame in the reference video and the description text can be collectively encoded by a pre-trained natural language supervised transferable visual model. In step S302, the prompt codes corresponding to each frame in the reference video can be counted by a k-neighbor algorithm. In this embodiment, each frame in the uploaded video and the description text can be jointly encoded and then classified based on the k-neighbor algorithm. The k-neighbor algorithm can be simply described as: if most of the k most similar samples (i.e., the most adjacent samples in the feature space) of a sample belong to a certain category, then the sample also belongs to this category. Accordingly, an abnormal event in the uploaded video can be characterized by a monitoring code that is similar to most prompt codes.

[0076] With the monitoring code, video monitoring can be performed in step S400.

[0077] In some optional implementations, comparing each frame code in the video stream with the monitoring code to determine whether the monitoring event occurs includes:

[0078] Step S401: when the similarity between any frame code in the video stream and the monitoring code exceeds a set value, the frame in the video stream is used as a key frame;

[0079] Step S402: When the length of the continuous frame sequence formed by the key frames exceeds a threshold, an abnormal event alarm behavior is triggered.

[0080] It is understandable that step S401 is used to determine the key frames of abnormal behavior, and in step S402, multiple key frames of abnormal behavior can represent an abnormal event. For example, the setting value is 80%, and the threshold is 150, which means that the similarity exceeds 80%, and 150 frames appear continuously (or the length of the continuous frame sequence is represented by time, such as 15s), then the monitoring event of abnormal behavior is determined.

[0081] In an alternative implementation manner, whether the monitoring event occurs may also be determined by the following steps:

[0082] Step S403: When the similarity between any frame code in the video stream and the monitoring code exceeds a first set value, the frame is used as the starting frame, and the threshold value M is calculated for the subsequent n consecutive frames using the following formula:

[0083]

[0084] Among them, SIM i is the similarity of the i-th frame, P is the second set value, and S(n) is the Sigmoid function.

[0085] Step S404: When the threshold value M is greater than 0, an abnormal event alarm behavior is triggered.

[0086] First, the first embodiment consisting of step S401-step S402 is explained. This embodiment shows that a key frame may have abnormal behavior. If this situation continues to occur, such as 150 frames mentioned in the above embodiment, it further indicates that the possibility of abnormal behavior is very high. In this embodiment, the setting value for judging the key frame is usually taken to be relatively large, such as the 80% given above. Only when the similarity exceeds 80% can it be identified as a key frame. However, there is a defect in this embodiment, that is, it is assumed that the similarity calculated for the first frame is very close to 100%. At this time, we believe that abnormal behavior has occurred, but the program will not confirm the abnormal behavior until the subsequent 150 frames are judged, and the judgment of the subsequent 150 frames may be unexpected, resulting in the failure of abnormal behavior identification. For this reason, the present application provides a second embodiment of steps S403-step S404.

[0087] In the second embodiment, step S403 introduces a first set value, which is usually much smaller than the set value in step S401, for example, 40%, indicating that when the similarity exceeds 40%, the threshold value M starts to be calculated.

[0088] It can be seen from step S404 that in step S403, actually, the comparison and The size of It is defined here as a continuous average similarity. For example, when n=1, the similarity of the first frame is 50%, and the average similarity is 50%. When n=2, if the similarity is still greater than 40%, for example, 60%, the average similarity is 55%. n has an upper limit, for example, 10. When M calculated for 10 consecutive times does not trigger the condition of step S404, the calculation process is released. It is defined as a dynamic threshold here. When n is in the range of 1-10, the Sigmoid function S(n) is roughly between 0.5-1 and is positively correlated with n. P is the second set value, which is a fixed value, such as 50%. After dividing it with the Sigmoid function S(n), its value changes dynamically between 100%-50% as n changes. Moreover, the larger n is, the lower the dynamic threshold is.

[0089] According to the above description, it can be seen that when the similarity of the first frame is particularly large, the average similarity calculated will be greater than the dynamic threshold Trigger an alarm, otherwise it is necessary to continue the calculation. As n increases, the dynamic threshold Getting smaller and smaller, if the average similarity If the dynamic threshold is exceeded, This triggers an alarm.

[0090] It should also be noted that embodiment 2 uses multi-threading for calculation. In step S403, when the similarity between the kth frame code in the video stream and the monitoring code exceeds the first set value, the threshold value M is calculated at the k frame, k+1 frame, k+2 frame...k+n frame according to the above embodiment 2. This process is performed by allocating thread 1 and releasing thread 1 when n reaches the upper limit. At the same time, for the k+1 frame, if its similarity with the monitoring code exceeds the first set value, it is necessary to open up another thread 2, and calculate the threshold value M of the k+1 to k+n+1 frames in thread 2, and so on. It is necessary to allocate memory execution space for n threads in advance.

[0091] Through the above method, this application guides users to define monitoring areas and special events through simple input, thereby generating accurate monitoring areas that meet user expectations and push more and more accurate special event videos and messages.

[0092] The second aspect of the present application provides an intelligent monitoring device based on human-computer interaction corresponding to the above method, the device comprising:

[0093] A monitoring area generating module, used for generating a monitoring area according to a click operation of acquiring the monitoring picture taken by the first camera device, and / or a given language prompt text;

[0094] An extended monitoring area generation module is used to match pixel feature points of an area in the monitoring picture taken by the second camera device that has the same semantic identifier as the monitoring area, and mark the matched area as the extended monitoring area;

[0095] A monitoring code generation module, used to generate a monitoring code from a reference video and a description text of a monitoring event, wherein the reference video includes the monitoring event;

[0096] The video monitoring module is used to monitor the video stream of the monitoring area and the extended monitoring area, compare each frame code in the video stream with the monitoring code, and determine whether the monitoring event occurs.

[0097] In some optional implementations, the monitoring area generation module includes:

[0098] An image encoding unit, used to encode the monitoring picture taken by the first camera device to generate an image encoding vector;

[0099] A semantic identification unit, used to perform image segmentation on the monitoring screen captured by the first camera device, and to perform semantic identification on each segmented area;

[0100] A semantic coding unit, used to encode pixels in each area of ​​the semantic marker to generate a semantic coding vector;

[0101] A superposition vector generating unit, used for superimposing the image coding vector and the semantic coding vector to form a superposition vector;

[0102] A unit for generating a coding vector to be matched, used for encoding the coordinates on the monitoring screen captured by the first camera device and / or the language prompt text to generate a coding vector to be matched;

[0103] The similarity calculation unit is used to perform a similarity calculation on the encoding vector to be matched and the superposition vector to obtain a monitoring area.

[0104] In some optional implementations, the monitoring code generation module includes:

[0105] A single-frame prompt code generation unit, used for collectively encoding each frame in the reference video and the description text to form a set of prompt codes;

[0106] The video monitoring code generation unit is used to collect statistics on the prompt codes corresponding to each frame in the reference video to generate the monitoring code.

[0107] In some optional implementations, the video monitoring module includes:

[0108] A key frame determination unit, configured to use any frame in the video stream as a key frame when the similarity between the frame code in the video stream and the monitoring code exceeds a set value;

[0109] The alarm behavior generating unit is used to trigger an abnormal event alarm behavior when the length of the continuous frame sequence formed by the key frames exceeds a threshold.

[0110] In a third aspect of the present application, a computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an intelligent monitoring method based on human-computer interaction.

[0111] In a fourth aspect of the present application, a readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the intelligent monitoring method based on human-computer interaction as described above. The computer-readable storage medium may be included in the device described in the above embodiment; or it may exist alone without being assembled into the device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the device, the data is processed according to the above method.

[0112] Reference below Figure 5 , which shows a schematic structural diagram of a computer device 400 suitable for implementing the embodiments of the present application. Figure 5 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0113] like Figure 5 As shown, the computer device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 are also stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0114] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

[0115] In particular, according to the embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above functions defined in the method of the present application are executed. It should be noted that the computer storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus or a device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0116] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] The modules or units involved in the embodiments described in this application may be implemented by software or hardware. The modules or units described may also be set in a processor, and the names of these modules or units do not constitute limitations on the modules or units themselves in some cases.

[0118] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An intelligent monitoring method based on human-computer interaction, used in a monitoring camera device, characterized in that: The method comprises: Generate a monitoring area according to a click operation of acquiring a monitoring picture taken by a first camera device and / or a language prompt text given by the first camera device; Perform pixel feature point matching on an area in the monitoring picture taken by the second camera device that has the same semantic identifier as the monitoring area, and mark the matched area as an extended monitoring area; Acquire a monitoring code generated from a reference video and a description text of a monitoring event, wherein the reference video includes the monitoring event; Video stream monitoring is performed on the monitoring area and the extended monitoring area, and each frame code in the video stream is compared with the monitoring code to determine whether the monitoring event occurs.

2. The intelligent monitoring method based on human-computer interaction according to claim 1, characterized in that: Generating a monitoring area according to a click operation of acquiring a monitoring screen captured by a first camera device and / or a language prompt text given by the first camera device includes: Encoding the monitoring picture taken by the first camera device to generate an image coding vector; Performing image segmentation on the monitoring screen captured by the first camera device, and semantically marking each segmented area; Encoding pixels in each area of ​​the semantic mark to generate a semantic encoding vector; Superimposing the image coding vector and the semantic coding vector to form a superimposed vector; Encoding the coordinates on the monitoring screen captured by the first camera device and / or the language prompt text to generate a coding vector to be matched; A similarity operation is performed between the encoding vector to be matched and the superposition vector to obtain a monitoring area.

3. The intelligent monitoring method based on human-computer interaction according to claim 1, characterized in that: Obtain surveillance code generated by reference video and description text of surveillance events, including: Collectively encoding each frame in the reference video and the description text to form a set of prompt codes; The prompt codes corresponding to each frame in the reference video are counted to generate the monitoring code.

4. The intelligent monitoring method based on human-computer interaction according to claim 1, characterized in that: Comparing each frame code in the video stream with the monitoring code to determine whether the monitoring event occurs, including: When the similarity between any frame code in the video stream and the monitoring code exceeds a set value, the frame in the video stream is used as a key frame; When the length of the continuous frame sequence formed by the key frames exceeds a threshold, an abnormal event alarm behavior is triggered.

5. An intelligent monitoring device based on human-computer interaction, characterized in that: The device comprises: A monitoring area generating module, used for generating a monitoring area according to a click operation of acquiring the monitoring picture taken by the first camera device, and / or a given language prompt text; An extended monitoring area generation module is used to match pixel feature points of an area in the monitoring picture taken by the second camera device that has the same semantic identifier as the monitoring area, and mark the matched area as the extended monitoring area; A monitoring code generation module, used to generate a monitoring code from a reference video and a description text of a monitoring event, wherein the reference video includes the monitoring event; The video monitoring module is used to monitor the video stream of the monitoring area and the extended monitoring area, compare each frame code in the video stream with the monitoring code, and determine whether the monitoring event occurs.

6. The intelligent monitoring device based on human-computer interaction according to claim 5, characterized in that: The monitoring area generation module includes: An image encoding unit, used to encode the monitoring picture taken by the first camera device to generate an image encoding vector; A semantic identification unit, used to perform image segmentation on the monitoring screen captured by the first camera device, and to perform semantic identification on each segmented area; A semantic coding unit, used to encode pixels in each area of ​​the semantic marker to generate a semantic coding vector; A superposition vector generating unit, used for superimposing the image coding vector and the semantic coding vector to form a superposition vector; A unit for generating a coding vector to be matched, used for encoding the coordinates on the monitoring screen captured by the first camera device and / or the language prompt text to generate a coding vector to be matched; The similarity calculation unit is used to perform a similarity calculation on the encoding vector to be matched and the superposition vector to obtain a monitoring area.

7. The intelligent monitoring device based on human-computer interaction according to claim 5, characterized in that: The monitoring code generation module comprises: A single-frame prompt code generation unit, used for collectively encoding each frame in the reference video and the description text to form a set of prompt codes; The video monitoring code generation unit is used to collect statistics on the prompt codes corresponding to each frame in the reference video to generate the monitoring code.

8. The intelligent monitoring device based on human-computer interaction according to claim 5, characterized in that: The video monitoring module comprises: A key frame determination unit, configured to use any frame in the video stream as a key frame when the similarity between the frame code in the video stream and the monitoring code exceeds a set value; The alarm behavior generating unit is used to trigger an abnormal event alarm behavior when the length of the continuous frame sequence formed by the key frames exceeds a threshold.

9. A computer device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent monitoring method based on human-computer interaction as described in any one of claims 1 to 4.

10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it is used to implement the intelligent monitoring method based on human-computer interaction as described in any one of claims 1 to 4.

Images