Method and device for performing event identification in monitoring video and electronic equipment

Abstract

The invention provides a method and device for performing event identification in a monitoring video, equipment and a storage medium, and the method comprises the steps: obtaining a target label set corresponding to the monitoring video; wherein the target label set comprises a plurality of target labels; determining a target tag corresponding to a retrieval target in the retrieval item in the target tag set as a retrieval target tag; judging whether a target event exists in the monitoring video or not according to the retrieval target label; and if it is determined that the target event exists in the monitoring video, outputting event description information corresponding to the target event. The retrieval method does not depend on a timeline any more, but identifies the event based on the retrieval item, so that the target and the event concerned by the user can be quickly and accurately retrieved, the retrieval efficiency can be improved, and the event identification experience of the user is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of video content recognition, and in particular to a method, device and electronic device for performing event recognition in a surveillance video. Background Art

[0002] Consumers who use security cameras often encounter the need to find related events of the target of interest. For example, when a package is delivered, it is generally necessary to know the time of delivery, which company's courier is, the specific location of the package, which suspicious people approached after the package was delivered, when the package was taken away, and other information. For the retrieval of related events of the target of interest (such as a package) similar to the above, users generally need to manually filter and view the recorded surveillance videos one by one in the surveillance video list, which often takes a lot of time to view related videos in order to restore multiple events that occurred in the past and the correlation between the events, resulting in very inconvenient event retrieval in the surveillance video and a very poor experience. Summary of the invention

[0003] In view of this, in order to solve the above-mentioned technical problem of inconvenience in event retrieval in surveillance videos, the present disclosure provides a retrieval method, device and equipment for event retrieval in surveillance videos.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for event recognition in a surveillance video is provided, the method comprising:

[0005] Obtain a target tag set corresponding to the surveillance video; wherein the target tag set includes multiple target tags;

[0006] Determine the target tag in the target tag set corresponding to the search target in the search matter as the search target tag;

[0007] According to the retrieval target tag, determining whether there is a target event in the surveillance video;

[0008] If it is determined that the target event exists in the surveillance video, the event description information corresponding to the target event is output.

[0009] In an optional embodiment, the search targets in the search item include a main search target and N associated search targets, where N is an integer greater than or equal to 0.

[0010] In an optional implementation, when the search target in the search item includes a main search target and at least one associated search target, the method for constructing the search item includes:

[0011] Selecting the subject search target;

[0012] At least one of the associated search targets associated with the main search target is selected to complete the construction of the search item.

[0013] In an optional implementation, the search target in the search item is selected in the following manner:

[0014] Based on the received first selection instruction acting on the search target library, the search target in the search matter is selected from the search target library.

[0015] In an optional embodiment, the retrieval method includes:

[0016] When any one of the search items does not exist in the search target library, based on the received second selection instruction acting on the video screen, the search target is selected from the video screen;

[0017] Performing small sample training based on the retrieval target selected in the video screen to obtain a target detection model; wherein the target detection model is used to retrieve the retrieval target selected in the video screen in the monitoring video;

[0018] The search target selected in the video screen is added to the search target library to update the search target library.

[0019] In an optional implementation, judging whether a target event exists in the surveillance video according to the retrieval target tag includes:

[0020] Get event feature configuration information;

[0021] According to the retrieval target tag and the event feature configuration information, it is determined whether there is the target event matching the event feature configuration information in the surveillance video.

[0022] In an optional implementation, if it is determined that the target event exists in the surveillance video, then outputting event description information corresponding to the target event includes:

[0023] If it is determined that the target event exists in the surveillance video, determining the event description information according to the timestamp in the surveillance video and the target event;

[0024] The event description information is sent to the target device.

[0025] In an optional implementation, the event description information includes at least one of the following:

[0026] Text description information, voice description information, image description information.

[0027] According to a second aspect of an embodiment of the present disclosure, a device for performing event recognition in a surveillance video is provided, the device comprising:

[0028] An acquisition module, used to acquire a target tag set corresponding to a surveillance video; wherein the target tag set includes a plurality of target tags;

[0029] A retrieval module, used to determine a target tag in the target tag set corresponding to the retrieval target in the retrieval item as a retrieval target tag;

[0030] It is also used to determine whether there is a target event in the surveillance video according to the retrieval target tag;

[0031] The output module is used to output event description information corresponding to the target event if it is determined that the target event exists in the surveillance video.

[0032] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising:

[0033] processor;

[0034] a memory for storing instructions executable by the processor;

[0035] The processor is configured to execute the method as described in the first aspect.

[0036] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: In the present disclosure, when searching and identifying the targets and events of interest, it is no longer dependent on the timeline, but a targeted search is directly performed based on the search items composed of the search targets. The search target may represent the role of the story line, that is, the method may search based on the story line, so that the target of interest to the user can be quickly and accurately retrieved, which can improve the efficiency of the search and enhance the user's experience in event recognition.

[0037] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0039] Figure 1 The present invention is a flowchart of a method for performing event recognition in a surveillance video according to an exemplary embodiment.

[0040] Figure 2 The present invention is a flowchart of a method for performing event recognition in a surveillance video according to an exemplary embodiment.

[0041] Figure 3 The invention is a block diagram of a device for performing event recognition in a surveillance video according to an exemplary embodiment.

[0042] Figure 4 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0043] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are only examples of devices and methods of some embodiments of the present disclosure.

[0044] The disclosed embodiment provides a method for event recognition in surveillance videos. In this method, when identifying search items and events, it no longer relies on the timeline, but directly performs targeted retrieval based on the search items composed of the search target. Among them, the search target can represent the role of the story line, that is, the method can perform retrieval based on the story line, so that the target that the user is concerned about can be retrieved quickly and accurately, which can improve the efficiency of retrieval and enhance the experience of event recognition.

[0045] In an exemplary embodiment, a method for event recognition in a surveillance video is provided, which can be applied to an electronic device. The electronic device can store the video, and it can be a camera device, a computer, a base station, etc., which is not limited to this. Figure 1 As shown, the method may include:

[0046] S110, obtaining a target tag set corresponding to the surveillance video; wherein the target tag set includes a plurality of target tags;

[0047] S120, determining a target tag in the target tag set corresponding to the search target in the search matter as a search target tag;

[0048] S130, judging whether there is a target event in the surveillance video according to the search target tag;

[0049] S140: If it is determined that a target event exists in the surveillance video, then event description information corresponding to the target event is output.

[0050] In step S110, the electronic device may be provided with an artificial intelligence (AI) function for performing target recognition in a video. The artificial intelligence function may be implemented based on a neural network model or in other ways, which is not limited thereto.

[0051] Among them, the artificial intelligence function can perform target recognition on the surveillance video, thereby retrieving the supported targets in the surveillance video and generating corresponding target tags (also called identification (ID)). All target tags retrieved in the surveillance video can constitute a target tag set, so that the electronic device obtains the target tag set corresponding to the surveillance video.

[0052] Among them, the targets that can be identified by artificial intelligence functions can be divided into low-level targets and high-level targets. The feature information contained in high-level targets is generally less than that contained in low-level targets. Low-level targets can refer to a category of targets. High-level targets can refer to specific targets under the above-mentioned category of targets. For example, a low-level target can be a person, and a high-level target can be a person wearing clothes of a specific color. For another example, a low-level target can be a person, and a high-level target can be a person named Zhang San. For another example, a low-level target can be a car, and a high-level target can be a black car. For another example, a low-level target can be an express package, and a high-level target can be an express package of express company A.

[0053] Among them, for low-level targets, they can be identified through target detection. Target detection can output the location of the identified low-level targets in the image or video. Target detection can include at least one of the following: package detection, human shape detection, face detection, pet detection, vehicle detection, etc. Of course, target retrieval can also include detection of other targets, which is not limited to this.

[0054] After low-level targets are detected in surveillance videos based on target detection, the detected targets can be marked, that is, target labels are set for the detected targets.

[0055] Among them, for high-level targets, they can be retrieved through target recognition. Target recognition can confirm the label of the retrieved high-level target in the image or video. That is, target recognition can extract the features of the target in the image or video, and then compare it with the pre-stored features to determine the label of the target. Target recognition may include at least one of the following: package recognition, face recognition, human shape recognition (ReID), human shape attribute recognition, vehicle attribute recognition, etc. Of course, target recognition may also include the recognition of other targets, which is not limited.

[0056] When a high-level target is identified in a surveillance video based on target recognition, the identified target can be marked and a target label can be set for the identified target and the video.

[0057] In step S120, after the electronic device obtains the target tag set corresponding to the surveillance video, it can search for the search target in the target tag set based on the search target in the search item, thereby determining the target tag corresponding to the search target in the target tag set as the search target tag.

[0058] For example, a search item may include two search targets, namely a first search target named "package" and a second search target named "courier". After the electronic device obtains the target tag set, it can determine the target tag marked as "package" in the target tag set as the search target tag of the first search target, and can determine the target tag marked as "courier" in the target tag set as the search target tag of the second search target.

[0059] Among them, the search items can be preset in the electronic device, or the electronic device can obtain it from other devices, without limitation. When the search items are preset in the electronic device, the search items can be set before the electronic device leaves the factory, or after the electronic device leaves the factory, without limitation. In addition, after the search items are set, they can be modified later to better meet different needs.

[0060] The search item may include at least one search target. The at least one search target may include a main search target and N associated search targets, where N is an integer greater than or equal to 0.

[0061] That is to say, a search item may include one search target, in which case the unique search target can be used as the main search target. A search item may also include two or more search targets, in which case the two or more search targets may include one main search target and at least one associated search target associated with the main search target.

[0062] It should be noted that the main search target can be a low-level target or a high-level target, and there is no limitation on this. The associated search target can be a low-level target or a high-level target, and there is no limitation on this.

[0063] In some embodiments,

[0064] The search matter may include a main search target and two associated search targets, wherein the main search target may be a low-level target, one associated search target may be a high-level target, and the other associated search target may be a low-level target.

[0065] In one embodiment,

[0066] The search matter may include a main search target and two associated search targets, wherein the main search target may be a high-level target, one associated search target may be a high-level target, and the other associated search target may be a low-level target.

[0067] In step S130, after the electronic device obtains the search target tag corresponding to the search item, it can analyze the surveillance video based on the search target tag to determine whether there is a target event that the user is concerned about in the surveillance video.

[0068] Among them, the event feature configuration information can be obtained first, and then according to the retrieval target tag and the event feature configuration information, it is determined whether the target event corresponding to the event feature configuration information exists in the monitoring video. Among them, the identification and retrieval of events can be achieved through state machine conversion, logical coherence, relationship classification between multiple targets, etc. Of course, the identification and retrieval of events can also be achieved through other methods, which are not limited to this.

[0069] Among them, the event feature configuration information can be preset in the electronic device, or it can be obtained by the electronic device from other devices, without limitation. When the event feature configuration information is preset in the electronic device, the event feature configuration information can be set before the electronic device leaves the factory, or it can be set after the electronic device leaves the factory, without limitation. In addition, after the event feature configuration information is set, it can be modified later to better meet different needs.

[0070] The event feature configuration information may include M event features, where M is an integer greater than or equal to 1. That is, the event feature configuration information may include at least one event feature. Each event feature may be used to characterize one or a class of events. That is, each event feature may be used to identify and judge an event or a class of events in a video.

[0071] Among them, the events that can be characterized by the event feature may include at least one of the following: a package delivery event, a package removal event, a person wandering event, a person returning home event, a person leaving home event, etc. Of course, the events that can be characterized by the event feature may also include other events, which are not limited to this. An event may also refer to a specific scene. For example, a package delivery event may also refer to a package delivery scene, and a person returning home event may also refer to a person returning home scene.

[0072] Among them, the event feature corresponding to the package delivery event may be the feature of "package from nothing to something". The event feature corresponding to the package removal event may be the feature of "package from something to nothing". The event feature corresponding to the character wandering event may be the feature of "people with the same label stay in the screen for a set time", and the set time can be set according to actual needs, and there is no limitation on this. The event feature corresponding to the character returning home event may be the "first behavior trajectory", and the first behavior trajectory may be the trajectory of moving from the first set position outside the first set house to the first set house, wherein the first set house and the first set position may be set according to actual needs, and there is no limitation on this. The event feature corresponding to the character leaving home event may be the "second behavior trajectory", and the second behavior trajectory may be the trajectory of moving from the second set house to the second set position outside the second set house, wherein the second set house and the second set position may be set according to actual needs, and there is no limitation on this.

[0073] It should be noted that, in addition to the event features introduced above, other event features may also be set, and there is no limitation to this.

[0074] In some embodiments,

[0075] The event feature configuration information includes an event feature for identifying a package delivery event, and the event feature may be "a package from nothing to something". The retrieval target tag may include a first retrieval target tag labeled "package" and a second retrieval target tag labeled "courier". In this embodiment, all video segments including at least one of the above two tags may be selected from the surveillance video, and then the video segment may be identified based on the event feature of "a package from nothing to something". If an event matching the above event feature is identified from the video segment, that is, if the event of "a package from nothing to something" is identified from the video segment, the identified event may be determined as a target event matching the above event feature configuration information.

[0076] In some embodiments,

[0077] The event feature configuration information includes a first event feature for identifying a package delivery event, and the first event feature may be "a package from nothing to something". The event feature configuration information may also include a second event feature for identifying a person returning home event, and the second event feature may be a "first path trajectory" that represents the person returning home. The event feature configuration information may also include a third event feature for identifying a package removal event, and the third event feature may be "a package from something to nothing".

[0078] The search target tag may include a first search target tag marked as "package" and a second search target tag marked as "Wang Wu", wherein the second search target is used as a main search target and the first search target is used as an associated search target.

[0079] In this implementation, all video segments including at least one of the above two labels can be selected from the surveillance video, and then the video segments can be identified based on the event features of "package from nothing to something", "first path trajectory" and "package from existence to nothing". If an event matching the above event features is identified from the video segment, that is, the events of "package from nothing to something", "first path trajectory" and "package from existence to nothing" are identified from the video segment, the identified event can be determined as a target event matching the above event feature configuration information.

[0080] It should be noted that, in addition to the above-mentioned implementation methods, event identification can also be performed in other ways, which are not limited to this.

[0081] In step S140, when a target event matching the event feature configuration information is retrieved in the surveillance video, it can be determined that there is a target event matching the event feature configuration information in the surveillance video, and event description information corresponding to the target event can be output. Semantic conversion can be performed based on the retrieved target tag, the target event, and the timestamp in the surveillance video to determine the event description information.

[0082] The event description information may include at least one of the following: text description information, voice description information, and image description information. Of course, the event description information may also include other types of information, which is not limited to better meet different needs.

[0083] In some embodiments,

[0084] The event description information may include text description information. In this embodiment, after the electronic device obtains the target event based on the surveillance video, it can convert the semantic description based on the search target tag, the timestamp of the surveillance video and the target event, thereby obtaining the text description information of the semantic description. Among them, in the text description information, the subject search target can be used as the subject in the text description information.

[0085] In some embodiments,

[0086] The event description information may include voice description information. In this embodiment, text description information may be obtained based on the above embodiment, and then the text description information may be converted into voice description information. In the voice description information, the subject search target may be used as the subject in the voice description information.

[0087] It should be noted that, in addition to determining the detection results in the above-mentioned manner, the target event may also be determined in other ways, which are not limited to better meet different needs.

[0088] Among them, after the electronic device obtains the event description information of the target event, it can send the event description information to the target device (such as the user's mobile phone) so that relevant personnel (such as users) can promptly understand the retrieval results of the target event they are concerned about, thereby improving the efficiency of relevant personnel in tracing the retrieval matters and target events they are concerned about and improving the user experience.

[0089] The target device can be set according to actual needs, and can be a user's mobile phone or wearable device, etc., without limitation. After the electronic device determines the event description information, it can transmit the event description information to the target device, so that the target device receives the event description information of the target event of interest, so that the user can understand the search results in a timely manner.

[0090] For example, the event description information may include text description information. After the electronic device obtains the text description information of the target event of concern, it can send the above text description information to the user's mobile phone in the form of a text message, so that the user can promptly understand the search results of the target event of concern.

[0091] For another example, the event description information may include voice description information. After the electronic device obtains the voice description information of the target event of concern, the voice description information can be pushed to the user's wearable device in the form of a network message, so that the user can promptly understand the retrieval results of the target event of concern.

[0092] It should be noted that, in addition to transmitting the event description information to the target device in the above manner, the search results may also be transmitted to the target device in other ways, which are not limited to this.

[0093] In some embodiments,

[0094] The event feature configuration information includes an event feature for identifying a package delivery event, and the event feature may be "a package comes into being from nothing". The retrieval target tag may include a first retrieval target tag labeled "package" and a second retrieval target tag labeled "courier". In this embodiment, if the above-mentioned package delivery event is identified from the surveillance video, a semantic conversion may be performed based on the first retrieval target tag labeled "package" and the second retrieval target tag labeled "courier", the identified package delivery event, and the timestamp in the surveillance video (for example, the time of delivering the package is 11:56), and the event description information is determined to be "the courier delivered the package at 11:56".

[0095] In some embodiments,

[0096] The event feature configuration information includes a first event feature for identifying a package delivery event, and the first event feature may be "a package from nothing to something". The event feature configuration information may also include a second event feature for identifying a person returning home event, and the second event feature may be a "first path trajectory" that represents the person returning home. The event feature configuration information may also include a third event feature for identifying a package removal event, and the third event feature may be "a package from something to nothing".

[0097] The search target tag may include a first search target tag marked as "package" and a second search target tag marked as "Wang Wu", wherein the second search target is used as a main search target and the first search target is used as an associated search target.

[0098] In this implementation, if a target event matching the above event feature configuration information is identified in the surveillance video, and the time displayed by the timestamp corresponding to the package delivery event is 10:30, and the time displayed by the timestamp corresponding to the person returning home event and the package taking away event is 18:00, then the event description information for the target event can be obtained as "Wang Wu took away the package delivered at 10:30 when he returned home at 18:00".

[0099] This method no longer relies on the timeline when searching for search items and events, but directly conducts targeted searches based on the search items composed of the search targets. The search targets can represent the characters of the story line, that is, the method can search based on the story line, so that the targets that the user is concerned about can be retrieved quickly and accurately, which can improve the efficiency of the search and enhance the user's search experience.

[0100] In addition, it should be noted that in this method, the monitoring video can be a video being recorded or a video that has been recorded. That is to say, the electronic device can retrieve the target event based on the retrieval item while recording the video, so that the relevant personnel can more realistically understand the relevant information of the target and event they are concerned about. The electronic device can also retrieve the target event based on the retrieval item for the video that has been recorded, so that the relevant personnel can more conveniently trace the relevant information of the target and event they are concerned about.

[0101] In an exemplary embodiment, a method for event recognition in a surveillance video is provided, which can be applied to an electronic device. Figure 2 As shown, in this method, when at least one search target includes a main search target and at least one associated search target, the method for constructing the search item may include:

[0102] S210, selecting a subject search target;

[0103] S220. Select at least one associated search target associated with the main search target to complete the construction of the search item.

[0104] In step S210, a search target library may be preset in the electronic device, and the search target library includes multiple search targets supported by the electronic device. In other words, the search target library may be composed of targets that the electronic device can search. The search target library may be set before the electronic device leaves the factory, or it may be set after the electronic device leaves the factory, and there is no limitation on this. After the search target library is set, it may be modified later to better meet different needs.

[0105] Among them, based on the received first selection instruction acting on the search target library, the search target in the search matter can be selected from the search target library.

[0106] When any search target in the search item does not exist in the search target library, that is, for the search target that does not exist in the search target library, this search target (that is, the search target that does not exist in the search target library) can be selected from the video screen based on the second selection instruction received and applied to the video screen.

[0107] Then, the retrieval target selected in the video screen can be used as a training target for small sample training to determine the target detection model corresponding to the retrieval target. The target detection model can be used to retrieve the target in the surveillance video. That is, the target detection model can be used to retrieve the target in the surveillance video later.

[0108] It should be noted that Local Few-shot Learning is a machine learning method that aims to solve the problem of building high-performance models when the data set is small or unbalanced. This method uses a small amount of data to train the model locally (i.e. on the device), thereby achieving fast and efficient model training and deployment.

[0109] In this step, when the main search target of the search matter is a target that already exists in the search target library, the relevant personnel can input a first selection instruction acting on the search target library into the electronic device. The first selection instruction can be used to select a search target in the search target library. After the electronic device receives the above-mentioned first selection instruction, it can determine the target selected by the first selection instruction in the search target library as the main search target based on the above-mentioned first selection instruction.

[0110] In some embodiments,

[0111] The search target library may include human figures, pets, packages, etc., which have been entered with IDs (also called tags, logos, names). The user may select a search target using a drop-down option in the application to select a main search target from the search target library.

[0112] When the main search target of the search matter does not exist in the search target library, the relevant personnel can input a second selection instruction acting on the video screen to the electronic device. The second selection instruction can be used to select a target that is the same as the above search target from the video screen. After the electronic device receives the above second selection instruction, it can select a target in the video screen based on the above second selection instruction, and then perform small sample training based on the above selected target to obtain a target detection model. In addition, the above selected target is added to the search target library to update the search target library. Then, the above search target is selected in the updated search target library based on the first selection instruction.

[0113] In one embodiment,

[0114] Retrieval targets that do not exist in the retrieval target library may include targets such as garden decorations, trash cans, or warehouse doors. Users can select such retrieval targets in the video screen of the real-time stream and add a name. Then, the data corresponding to the above retrieval targets can be used as data for small sample training, and a target detection model can be generated after local small sample training. The target detection model enables electronic devices to detect the above targets in surveillance videos. At the same time, the above targets are added to the retrieval target library to update the retrieval target library, so that the above targets can be selected as retrieval targets in the retrieval target library later.

[0115] It should be noted that, in addition to being selected in the above-mentioned manner, the subject search target can also be determined in other ways, and there is no limitation to this.

[0116] In step S220, after the electronic device determines the main search target of the search item, if relevant personnel need to configure an associated search target for the above-mentioned main search target, the electronic device can determine at least one associated search target associated with the main search target in a similar manner to complete the construction of the search item.

[0117] Among them, the method of determining the associated search target can refer to the method of determining the main search target, which will not be elaborated here.

[0118] It should be noted that, in addition to selecting related search targets in the above-mentioned manner, related search targets may also be selected in other manners, which are not limited thereto.

[0119] In addition, when the search items are constructed, the search items can be saved to the electronic device, and the electronic device can then actively search based on the search items. Of course, after the search items are saved to the electronic device, the electronic device can also search based on the search items based on the selection of the relevant personnel. For example, if the relevant personnel need to continue to search the search items, they can directly determine the search items as the search items used by the electronic device to quickly search the search items.

[0120] In the method, by constructing a search item including a main search target and an associated search target, the search for the targets and events of interest in the surveillance video can be more accurately performed based on the story line, thereby quickly obtaining the targets and events of interest to the user. Moreover, since there is a primary and secondary distinction between the search targets in the search items (i.e., the main search target and the associated search target), it is easier for the user to search the relationship between the main search target and the associated search target, and it is easier for the user to understand the relationship between the targets and events of interest to the user, thereby improving the user experience.

[0121] In an exemplary embodiment, a method for event recognition in a surveillance video is provided, which can be applied to an electronic device. Assuming that a user is concerned about the event of driving away from home every day, user Zhang San can set himself as the main search target and the vehicle as the associated search target to form a search item including Zhang San and the vehicle. Among them, Zhang San is the main search target, which is a high-level target and can be retrieved by face recognition. The vehicle is an associated search target, which is a low-level target and can be retrieved by vehicle detection. The event feature configuration information may include the event feature of the vehicle driver's seat changing from no one to someone.

[0122] In this implementation, the electronic device can simultaneously determine a target tag set corresponding to the recorded video during the process of recording a video, and then search the target tag set based on the retrieval items to obtain the tags "Zhang San" and "vehicle". Then, the electronic device can judge the event characteristics of "the driver's seat of the vehicle changes from no one to someone" in the recorded video based on the above-mentioned "Zhang San" and "vehicle" tags.

[0123] If, based on the tags "Zhang San" and "vehicle", an event corresponding to the above event features is determined in the recorded video, that is, the event feature of the driver's seat of "vehicle" changing from no one to someone is retrieved, it can be determined that the event of Zhang San driving away from home is retrieved, and the time when Zhang San drove away from home (for example, 12:00) is obtained based on the timestamp, and then the retrieved information is semantically converted based on logical conjunctions to obtain the event description information of the target event. The event description information can be "Zhang San drove away from home at 12:00".

[0124] After the electronic device obtains the event description information, it can transmit the event description information to other devices bound to the electronic device, so that users of the other devices (such as Zhang San himself, or Zhang San’s wife, etc.) can promptly learn about Zhang San’s driving away from home.

[0125] In an exemplary embodiment, a retrieval method is provided, which can be applied to an electronic device. Assuming that a user is concerned about the delivery and pickup of a package, the user Zhang San can set the package as the main retrieval target, and can set the person as the associated retrieval target to form a retrieval item including the package and the person. Among them, the package and the person are both low-level targets, and the package can be retrieved by package detection, and the person can be retrieved by human shape detection. The event feature configuration information can include the event feature of the package from non-existence to existence, and the event feature of the package from existence to non-existence.

[0126] In this implementation, the recorded video can be searched based on the artificial intelligence function to determine the target tag set corresponding to the above video, and then the target tag set can be searched based on the search items to obtain the tags "package" and "person". Then, the electronic device can search for the two event features of "package from nothing to something" and "package from something to nothing" in the above video based on the above "package" and "person" tags.

[0127] If it is determined in the video that there is an event corresponding to the event feature of "the package comes from nothing", that is, the event feature of "the package" coming from nothing to something is retrieved, it can be determined that the event of package delivery is retrieved, and the time of package delivery (for example, 8:00) is obtained based on the timestamp in the video, and then the retrieved information is semantically converted based on logical conjunctions to obtain the event description information of the target event corresponding to the event feature of "the package comes from nothing". The event description information can be "the package was delivered by someone at 8:00". After the electronic device obtains the event description information, it can transmit the above event description information to Zhang San's mobile phone, so that Zhang San can learn about the relevant information of the package delivery in a timely manner.

[0128] If the event corresponding to the event feature of "the package changes from existence to non-existence" is determined in the video, that is, the event feature of "the package" changes from existence to non-existence is retrieved, it can be determined that the event of the package being taken away is retrieved, and the time of the package being taken away (for example, 15:00) is obtained based on the timestamp in the video, and then the retrieved information is semantically converted based on logical conjunctions to obtain the event description information of the target event corresponding to the event feature of "the package changes from existence to non-existence". The event description information can be "the package was taken away by someone at 15:00". After the electronic device obtains the event description information, it can transmit the above event description information to Zhang San's mobile phone, so that Zhang San can promptly understand the relevant information about the package being taken away.

[0129] It should be noted that, in addition to being used for the above-mentioned search items and events, the method of the present application can also be used for the search of other search items and events, without limitation thereto.

[0130] In an exemplary embodiment, a device for event recognition in a surveillance video is provided, which can be applied to an electronic device. The device can be used to implement the above method. For example, referring to Figure 3 As shown, the device may include an acquisition module 10 , a retrieval module 20 and an output module 30 .

[0131] The acquisition module 10 is used to acquire a target tag set corresponding to the surveillance video; wherein the target tag set includes multiple target tags;

[0132] A retrieval module 20, configured to determine a target tag in the target tag set corresponding to a retrieval target in a retrieval item as a retrieval target tag;

[0133] It is also used to determine whether there is a target event in the surveillance video based on the retrieval target label;

[0134] The output module 30 is used to output event description information corresponding to the target event if it is determined that the target event exists in the surveillance video.

[0135] In an exemplary embodiment, a device for event recognition in surveillance video is provided, which can be applied to electronic devices. In the device, the search target in the search item includes a main search target and N associated search targets, where N is an integer greater than or equal to 0.

[0136] In an exemplary embodiment, a device for identifying events in a surveillance video is provided, which can be applied to an electronic device. Figure 3 As shown, in the device, when the search target in the search item includes a main search target and at least one associated search target, the search module 20 constructs the search item by the following method:

[0137] Select the subject search target;

[0138] At least one associated search target associated with the main search target is selected to complete the construction of the search item.

[0139] In an exemplary embodiment, a device for event recognition in a surveillance video is provided, which can be applied to an electronic device. In the device, a retrieval module 20 can be used to:

[0140] Based on the received first selection instruction acting on the search target library, a search target in the search matter is selected from the search target library.

[0141] In an exemplary embodiment, a device for identifying events in a surveillance video is provided, which can be applied to an electronic device. Figure 3 As shown, in the device, the retrieval module 20 can be used to:

[0142] When any one of the search targets in the search item does not exist in the search target library, based on the received second selection instruction acting on the video screen, the search target is selected from the video screen;

[0143] Performing small sample training based on the retrieval target selected in the video screen to obtain a target detection model; wherein the target detection model is used to retrieve the retrieval target selected in the video screen in the monitoring video;

[0144] The search target selected in the video screen is added to the search target library to update the search target library.

[0145] In an exemplary embodiment, a device for identifying events in a surveillance video is provided, which can be applied to an electronic device. Figure 3 As shown, in the device,

[0146] An acquisition module 10, which can be used to acquire event feature configuration information;

[0147] The retrieval module 20 can be used to determine whether there is a target event matching the event feature configuration information in the surveillance video according to the retrieval target tag and the event feature configuration information.

[0148] In an exemplary embodiment, a device for identifying events in a surveillance video is provided, which can be applied to an electronic device. Figure 3 As shown, in the device, the output module 30 can be used to:

[0149] If it is determined that the target event exists in the surveillance video, the event description information is determined according to the timestamp and the target event in the surveillance video;

[0150] Send event description information to the target device.

[0151] In an exemplary embodiment, a device for event recognition in a surveillance video is provided, which can be applied to an electronic device. In the device, the event description information includes at least one of the following:

[0152] Text description information, voice description information, image description information.

[0153] In an exemplary embodiment, an electronic device is provided. The electronic device can store videos. The electronic device can include a camera device, a computer, a smart home, a smart appliance, a base station, etc., without limitation.

[0154] Among them, reference Figure 4 As shown, the electronic device 500 includes: at least one processor 501, a memory 502, at least one network interface 504 and a user interface 503. The various components in the electronic device 500 are coupled together through a bus system 505. It can be understood that the bus system 505 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity of the description, various buses are marked as bus system 505 in the figure.

[0155] The user interface 503 may include a display, a keyboard or a pointing device (eg, a mouse, a trackball), a touch pad or a touch screen.

[0156] It can be understood that the memory 502 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0157] In some implementations, the memory 502 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 5021 and an application program 5022 .

[0158] Among them, the operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 5022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present application can be included in the application 5022.

[0159] In the embodiment of the present application, by calling the program or instruction stored in the memory 502, specifically, the program or instruction stored in the application 5022, the processor 501 is used to execute the method steps provided by each method embodiment, for example, including:

[0160] Obtain a target tag set corresponding to the surveillance video; wherein the target tag set includes multiple target tags;

[0161] Determine the target tag in the target tag set corresponding to the search target in the search matter as the search target tag;

[0162] According to the retrieval target label, determine whether there is a target event in the surveillance video;

[0163] If it is determined that a target event exists in the surveillance video, the event description information corresponding to the target event is output.

[0164] The method disclosed in the above embodiment of the present application can be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 501. The above processor 501 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and completes the steps of the above method in combination with its hardware.

[0165] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of the present application, or a combination thereof.

[0166] For software implementation, the technology of this article can be implemented by a unit that performs the functions of this article. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0167] The terminal provided in this embodiment can execute all the steps of the above-mentioned data processing method, thereby achieving the technical effect of the above-mentioned data processing method. Please refer to the relevant description of the above-mentioned data processing method for details. For the sake of concise description, it will not be repeated here.

[0168] The embodiment of the present application also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0169] When one or more programs in the storage medium can be executed by one or more processors, the data processing method executed on the electronic device side can be implemented.

[0170] The processor is used to execute the program stored in the memory to implement the following steps of the data processing method performed on the electronic device side:

[0171] Obtain a target tag set corresponding to the surveillance video; wherein the target tag set includes multiple target tags;

[0172] Determine the target tag in the target tag set corresponding to the search target in the search matter as the search target tag;

[0173] According to the retrieval target label, determine whether there is a target event in the surveillance video;

[0174] If it is determined that a target event exists in the surveillance video, the event description information corresponding to the target event is output.

[0175] The professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0176] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0177] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for event recognition in surveillance video, It is characterized in that The method comprises: Obtain a target tag set corresponding to the surveillance video; wherein the target tag set includes multiple target tags; Determine the target tag in the target tag set corresponding to the search target in the search matter as the search target tag; According to the retrieval target tag, determining whether there is a target event in the surveillance video; If it is determined that the target event exists in the surveillance video, the event description information corresponding to the target event is output.

2. The method according to claim 1, It is characterized in that The search targets in the search item include a main search target and N associated search targets, where N is an integer greater than or equal to 0.

3. The method according to claim 1, It is characterized in that When the search target in the search item includes a main search target and at least one associated search target, the method for constructing the search item includes: Selecting the subject search target; At least one of the associated search targets associated with the main search target is selected to complete the construction of the search item.

4. The method according to claim 1, It is characterized in that The search target in the search item is selected in the following way: Based on the received first selection instruction acting on the search target library, the search target in the search matter is selected from the search target library.

5. The method according to claim 4, It is characterized in that The retrieval method comprises: When any one of the search items does not exist in the search target library, based on the received second selection instruction acting on the video screen, the search target is selected from the video screen; Performing small sample training based on the retrieval target selected in the video screen to obtain a target detection model; wherein the target detection model is used to retrieve the retrieval target selected in the video screen in the monitoring video; The search target selected in the video screen is added to the search target library to update the search target library.

6. The method according to claim 1, It is characterized in that The step of determining whether a target event exists in the surveillance video according to the retrieval target tag includes: Get event feature configuration information; According to the retrieval target tag and the event feature configuration information, it is determined whether there is the target event matching the event feature configuration information in the surveillance video.

7. The method according to claim 6, It is characterized in that If it is determined that the target event exists in the surveillance video, then outputting event description information corresponding to the target event includes: If it is determined that the target event exists in the surveillance video, determining the event description information according to the timestamp in the surveillance video and the target event; The event description information is sent to the target device.

8. The method according to any one of claims 1 to 7, It is characterized in that The event description information includes at least one of the following: Text description information, voice description information, image description information.

9. A device for identifying events in surveillance video, It is characterized in that The device comprises: An acquisition module, used to acquire a target tag set corresponding to a surveillance video; wherein the target tag set includes a plurality of target tags; A retrieval module, used to determine a target tag in the target tag set corresponding to the retrieval target in the retrieval item as a retrieval target tag; It is also used to determine whether there is a target event in the surveillance video according to the retrieval target tag; The output module is used to output event description information corresponding to the target event if it is determined that the target event exists in the surveillance video.

10. An electronic device, It is characterized in that The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the method according to any one of claims 1 to 8.

Images