Data processing method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention discloses a data processing method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a home security video collected by at least one camera in a preset time period through a server, carrying out the event recognition processing of the home security video, and obtaining the event description information of at least one event, and performing event summarization processing on the event description information based on the event processing large model to obtain important event report information, and sending the important event report information to the terminal, so that the terminal displays the important event report information on the smart home management interface.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, storage medium, and electronic device. Background Art

[0002] With the development of smart home technology, an increasing number of smart home devices have emerged, bringing significant convenience to people's daily lives. Some of these devices are designed to safeguard home security. For example, outdoor smart cameras, smart doorbells, and smart door locks can capture real-time video of the outdoor environment and upload it to the cloud, where corresponding cloud-based software pushes the video to users. Alternatively, indoor smart cameras can capture real-time video of the indoor environment, allowing users to use cloud-based software to monitor indoor activities. Therefore, users can use the video captured by these cameras to promptly monitor unusual events in their home environment, enabling them to implement timely safety measures to address these incidents. Summary of the Invention

[0003] The embodiments of the present application provide a data processing method, apparatus, computer storage medium, and electronic device. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a data processing method, applied to a server, the method comprising:

[0005] Acquiring a home security video captured by at least one camera within a preset time period, and performing event recognition processing on the home security video to obtain event description information of at least one event;

[0006] Based on the event processing model, the event description information is aggregated to obtain important event report information;

[0007] The important event report information is sent to the terminal, so that the terminal displays the important event report information on the smart home management interface.

[0008] In a possible implementation, the event processing macro model is used to aggregate the event description information to obtain important event report information, including:

[0009] Based on the event processing macro model, event screening processing is performed on the event description information to obtain target event description information, and event aggregation processing is performed on the target event description information to obtain important event report information.

[0010] In a possible implementation, the performing event screening processing on the event description information based on the event processing macro model to obtain target event description information, and performing event aggregation processing on the target event description information to obtain important event report information, include:

[0011] Determining a reporting priority of the at least one event, and generating an event processing prompt word based on the reporting priority and the event description information;

[0012] Based on the event processing prompt word, the event processing large model is used to perform event screening processing on the event description information to obtain target event description information, and event aggregation processing is performed on the target event description information to obtain important event report information.

[0013] In a possible implementation manner, determining the reporting priority of the at least one event includes:

[0014] Obtaining preset event information corresponding to the user, and determining a first reporting priority of the at least one event based on the preset event information; and / or,

[0015] Acquire historical video access behaviors of a user, determine preferred event information corresponding to the user based on the historical video access behaviors, and determine a second reporting priority of the at least one event based on the preferred event information.

[0016] In a possible implementation manner, before sending the important event report information to the terminal, the method further includes:

[0017] Obtaining a target video clip corresponding to the important event report information, generating a video jump link based on the target video clip, and performing data splicing processing on the important event report information and the video jump link to obtain comprehensive important event report information;

[0018] The sending of the important event report information to the terminal so that the terminal displays the important event report information on the smart home management interface includes:

[0019] The important event comprehensive report information is sent to the terminal, so that the terminal displays the important event comprehensive report information on the smart home management interface.

[0020] In a possible implementation, performing event recognition processing on the home security video to obtain event description information of at least one event includes:

[0021] Performing person identification processing on the home security video to obtain the identity of the target person, and performing person behavior identification processing on the home security video to obtain the behavior of the target person;

[0022] Event description information of at least one event is determined based on the target person's identity and the target person's behavior.

[0023] In a second aspect, an embodiment of the present application provides a data processing method, applied to a terminal, the method comprising:

[0024] Obtaining important event report information sent by a server, the important event report information being obtained by the server performing event aggregation processing on event description information of at least one event based on an event processing macromodel, the event description information being obtained by the server performing event recognition processing based on home security video captured by at least one camera;

[0025] The important event report information is displayed on the smart home management interface.

[0026] In a possible implementation, displaying the important event report information on the smart home management interface includes:

[0027] Determine the event reporting area in the smart home management interface;

[0028] The important event report information is displayed in the event report area.

[0029] In a possible implementation manner, the method further includes:

[0030] In response to an event information setting operation input by a user, a preset event information configuration interface is displayed;

[0031] The preset event information of interest input by the user for the preset event information configuration interface is obtained, and the preset event information of interest is sent to the server. The preset event information of interest is used by the server to generate event processing prompt words based on the event description information, and to use the event processing big model to perform event screening processing on the event description information based on the event processing prompt words to obtain target event description information, and to perform event summary processing on the target event description information to obtain important event report information.

[0032] In a third aspect, an embodiment of the present application provides a data processing device, applied to a server, comprising:

[0033] a video acquisition module, configured to acquire home security videos captured by at least one camera within a preset time period, and perform event recognition processing on the home security videos to obtain event description information of at least one event;

[0034] An event processing module is used to perform event aggregation processing on the event description information based on the event processing model to obtain important event report information;

[0035] The information sending module is used to send the important event report information to the terminal, so that the terminal displays the important event report information on the smart home management interface.

[0036] Optionally, the event processing module includes:

[0037] The event processing unit is used to perform event screening processing on the event description information based on the event processing model to obtain target event description information, and to perform event aggregation processing on the target event description information to obtain important event report information.

[0038] Optionally, the event processing unit includes:

[0039] a first event processing subunit, configured to determine a reporting priority of the at least one event, and generate an event processing prompt word based on the reporting priority and the event description information;

[0040] The second event processing sub-unit is used to perform event screening processing on the event description information based on the event processing prompt word using the event processing large model to obtain target event description information, and to perform event aggregation processing on the target event description information to obtain important event report information.

[0041] Optionally, the first event processing subunit includes:

[0042] A first priority determination subunit is configured to obtain preset event information corresponding to a user, and determine a first reporting priority of the at least one event based on the preset event information; and / or

[0043] The second priority determination subunit is used to obtain the user's historical video access behavior, determine the preference event information corresponding to the user based on the historical video access behavior, and determine the second reporting priority of the at least one event based on the preference event information.

[0044] Optionally, the device further includes:

[0045] An event splicing module is used to obtain a target video clip corresponding to the important event report information, generate a video jump link based on the target video clip, and perform data splicing processing on the important event report information and the video jump link to obtain comprehensive important event report information;

[0046] The information sending module is used to send the important event comprehensive report information to the terminal, so that the terminal displays the important event comprehensive report information on the smart home management interface.

[0047] Optionally, the video acquisition module includes:

[0048] a video recognition unit, configured to perform person identification processing on the home security video to obtain the identity of a target person, and perform person behavior identification processing on the home security video to obtain the behavior of the target person;

[0049] An event generating unit is configured to determine event description information of at least one event based on the target person's identity and the target person's behavior.

[0050] In a fourth aspect, an embodiment of the present application provides a data processing device, applied to a terminal, the device comprising:

[0051] an information acquisition module, configured to acquire important event report information sent by a server, the important event report information being obtained by the server performing event aggregation processing on event description information of at least one event based on an event processing macromodel, the event description information being obtained by the server performing event recognition processing based on home security video captured by at least one camera;

[0052] The event display module is used to display the important event report information on the smart home management interface.

[0053] Optionally, the event display module includes:

[0054] A first display processing unit is used to determine an event reporting area in the smart home management interface;

[0055] The second display processing unit is configured to display the important event report information in the event report area.

[0056] Optionally, the device further includes:

[0057] An event configuration module, configured to display a preset event information configuration interface in response to an event information setting operation input by a user;

[0058] A data sending module is used to obtain the preset event information of interest input by the user for the preset event information configuration interface, and send the preset event information of interest to the server. The preset event information of interest is used by the server to generate event processing prompt words based on the event description information, and to use the event processing big model to perform event screening processing on the event description information based on the event processing prompt words to obtain target event description information, and to perform event summary processing on the target event description information to obtain important event report information.

[0059] In a fifth aspect, an embodiment of the present application provides a computer storage medium, which has multiple instructions, and the instructions are suitable for being loaded by a processor and executing the above method.

[0060] In a sixth aspect, an embodiment of the present application provides an electronic device, which may include: a memory and a processor; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the memory and executing the above method.

[0061] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0062] In an embodiment of the present application, a server obtains home security videos collected by at least one camera within a preset time period, performs event recognition processing on the home security videos to obtain event description information of at least one event, performs event aggregation processing on the event description information based on the event processing big model to obtain important event report information, and sends the important event report information to the terminal so that the terminal can display the important event report information on the smart home management interface. In this way, the server identifies the event description information corresponding to at least one event in the home security videos collected by multiple devices, and then uses the powerful text generation capability of the big model to aggregate the events collected by multiple devices according to the event description information to obtain important event report information, and sends the important event report information to the terminal, so that the user of the terminal can quickly understand the information of important events that occurred in the home environment videos collected by multiple devices based on the important event report information, without the user having to view multiple videos of multiple devices one by one to obtain information about important events that occurred in the home environment within the preset time period. Therefore, the embodiment of the present application increases the convenience of users viewing important event information in home environment videos and improves the efficiency of users viewing important event information. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 This is a schematic diagram of a data processing system provided in an embodiment of the present application;

[0065] Figure 2 This is a flow chart of a data processing method provided in an embodiment of the present application;

[0066] Figure 3 This is a flow chart of another data processing method provided in an embodiment of the present application;

[0067] Figure 4 This is a flow chart of another data processing method provided in an embodiment of the present application;

[0068] Figure 5 This is a structural diagram of another data processing method provided in an embodiment of the present application;

[0069] Figure 6 This is a schematic diagram showing important event reporting information provided by an embodiment of the present application;

[0070] Figure 7 is a structural diagram of a data processing device provided in an embodiment of the present application;

[0071] Figure 8 This is a structural diagram of another data processing device provided in an embodiment of the present application;

[0072] Figure 9 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the purpose, features, and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0074] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In the description of this application, it should be noted that, unless otherwise expressly specified and limited, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances. In addition, in the description of this application, unless otherwise specified, "multiple" refers to two or more. "and / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0075] In related technologies, smart home devices such as smart cameras can capture videos from home environments and upload them to the cloud, allowing users to view the videos in real time through software used to manage smart home devices. However, the videos uploaded to the cloud through cameras can only detect videos corresponding to simple events, such as "screen changes" and "loud noises," and display the videos corresponding to these events to users for easy access. Because many events occur in a day or within a certain time period, potentially hundreds of events, users need to view the videos of individual events one by one to obtain information about important events that occurred within the preset time period. This makes it cumbersome for users to obtain important event information.

[0076] In order to solve the above technical problems, the present application is described in detail below with reference to specific embodiments.

[0077] See Figure 1 , is a schematic diagram of a data processing system provided in an embodiment of this specification. Figure 1 As shown, the scenario diagram may include at least a terminal cluster and a server.

[0078] In some embodiments, the terminal cluster may include at least one terminal, such as Figure 1 As shown, it specifically includes terminal 1 corresponding to user 1, terminal 2 corresponding to user 2, ..., terminal n corresponding to user n, where n is an integer greater than 0.

[0079] Each terminal in the terminal cluster can be a smart device with communication capabilities, including but not limited to wearable devices, handheld devices, personal computers, tablet computers, smartphones, computing devices, or other processing devices connected to a wireless modem. Smart devices can be called different names in different networks, such as user equipment, access terminal, subscriber unit, subscriber station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), electronic device in 5G network or future evolution network, etc.

[0080] In some embodiments, a server is a hardware device with strong computing capabilities. Specifically, the server can use a separate server device, such as a rack-mounted, blade, tower, or cabinet-mounted server device, or a workstation, mainframe computer, or other hardware device. A server cluster composed of multiple servers can also be used. The servers in the service cluster can be composed in a symmetrical manner, where each server has equivalent functions and status in the transaction link, and each server can provide services to the outside world independently. Providing services independently can be understood as not requiring the assistance of additional servers.

[0081] In some embodiments, the electronic device executing the data processing method may be a server, and the server may establish a communication connection with a terminal in the terminal cluster, and data exchange during the data processing process is completed based on the communication connection. For example, in the data processing method, the server obtains home security video captured by at least one camera within a preset time period, performs event recognition processing on the home security video to obtain event description information of at least one event, performs event aggregation processing on the event description information based on the event processing macro model to obtain important event report information, and sends the important event report information to the terminal, so that the terminal displays the important event report information on the smart home management interface.

[0082] It should be noted that the server and the terminal establish a communication connection through a network for interactive communication, wherein the network can be a wireless network or a wired network. Wireless networks include but are not limited to cellular networks, wireless local area networks, infrared networks, or Bluetooth networks, and wired networks include but are not limited to Ethernet, universal serial bus (USB), or controller area network. In one or more embodiments of the specification, technologies and / or formats including Hypertext Markup Language (HTML) and Extensible Markup Language (XML) are used to represent data exchanged over the network (such as target compressed packages). In addition, conventional encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0083] The data processing system embodiments provided in this specification share the same concept as the data processing methods described in one or more embodiments. The execution entity corresponding to the data processing methods described in one or more embodiments of this specification may be an electronic device, which may be the aforementioned server or terminal. The specific execution entity may be determined based on the actual application scenario. The specific implementation process of the data processing system embodiments can be found in the following method embodiments and will not be repeated here.

[0084] In one embodiment, Figure 2As shown, a data processing method is proposed. The method can be implemented by a computer program and can be run on a data processing device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone tool application.

[0085] Specifically, the data processing method is executed by a server, and the method includes:

[0086] S101: Acquire home security videos collected by at least one camera within a preset time period, and perform event recognition processing on the home security videos to obtain event description information of at least one event.

[0087] Among them, the camera refers to a smart camera installed in a home environment. The camera may specifically include but is not limited to cameras installed indoors, cameras installed around entrance doors (such as cameras in video doorbells or smart doorbells, cameras in smart door locks, or separate camera devices). The image acquisition range of each camera installed in the home environment may be different.

[0088] Optionally, the cameras in the embodiment of the present application may be multiple cameras installed in a certain family, or multiple cameras with device association relationships installed in different families with a family relationship. For example, the cameras in the embodiment of the present application may include a camera installed in the living room of "my home" and a camera installed around the entrance door of "my home" for collecting the environment outside the door; for another example, the cameras in the embodiment of the present application may include a camera installed in the living room of "my home", a camera installed around the entrance door of "my home" for collecting the environment outside the door, a camera installed around the entrance door of "my grandmother's home" for collecting the environment outside the door, and a camera installed in the living room of "my grandmother's home".

[0089] The preset time period may refer to a time period of several hours in the past, such as this morning, this afternoon, or from 8 am to 6 pm this morning.

[0090] It is understood that home security video refers to video captured in a home environment. When there is only one camera, home security video is the video captured by that camera; when there are multiple cameras, home security video can include the video captured by all cameras.

[0091] It is understood that event description information refers to descriptive information related to an event generated based on the behavior or activity trajectory of a person or an animal in a home environment. For example, the event description information may be in the form of: what a person did and when.

[0092] In some embodiments, obtaining home security videos captured by at least one camera within a preset time period may specifically include: obtaining home security videos captured within the preset time period and uploaded by at least one camera with a device association; or obtaining home security videos captured within the preset time period from a video database of at least one camera with a device association, where the videos in the video database are autonomously uploaded by the camera. It is understood that the at least one camera with a device association may be determined by an identifier of a camera pre-bound by the user on the user terminal.

[0093] The step of performing event recognition processing on the home security video to obtain event description information of at least one event can be: performing person recognition processing on the home security video to obtain event description information of at least one event. The person recognition processing can specifically include person identity recognition, person behavior recognition, and person activity trajectory recognition.

[0094] S102: performing event aggregation processing on the event description information based on the event processing big model to obtain important event report information.

[0095] Among them, the event processing large model can directly use the basic large language model (Large Language Model, LLM), or it can be a large language model obtained after model training for event processing scenarios based on the basic large language model.

[0096] Important event report information refers to report information in text form generated by summarizing important events in the event description information.

[0097] In some embodiments, the event description information is subjected to event screening processing through the event processing big model to obtain target event description information, and then the target event description information is subjected to event aggregation processing to obtain important event report information.

[0098] S103: Send the important event report information to the terminal, so that the terminal displays the important event report information on the smart home management interface.

[0099] In some embodiments, after generating the important event report information, the server immediately sends the important event report information to the terminal. Alternatively, after generating the important event report information, the server sends the important event report information to the terminal when a preset time arrives.

[0100] In an embodiment of the present application, a server obtains a home security video captured by at least one camera within a preset time period, performs event recognition processing on the home security video to obtain event description information of at least one event, performs event aggregation processing on the event description information based on the event processing big model to obtain important event report information, and sends the important event report information to the terminal so that the terminal can display the important event report information on the smart home management interface. In this way, the server identifies the event description information corresponding to at least one event in the home security video, and then uses the powerful text generation capability of the big model to generate important event report information based on the event description information, and sends the important event report information to the terminal, so that the user of the terminal can quickly understand the information of important events that occurred in the home environment video captured by multiple devices based on the important event report information, without the user having to view multiple videos of multiple devices one by one to obtain the information of important events that occurred in the home environment within the preset time period. Therefore, the embodiment of the present application increases the convenience of users viewing important event information in the home environment video and also improves the efficiency of users viewing important event information.

[0101] See Figure 3 , Figure 3 This is a flow chart of another embodiment of a data processing method proposed in this application.

[0102] Specifically, the data processing method is executed by a server, and the method includes:

[0103] S201: Acquire home security videos collected by at least one camera within a preset time period.

[0104] Specifically, the implementation of step S201 can be found in Figure 2 The description of the relevant steps in the illustrated embodiment will not be repeated in detail here.

[0105] S202 , performing person identification processing on the home security video to obtain the identity of the target person, and performing person behavior identification processing on the home security video to obtain the behavior of the target person.

[0106] The target person identity refers to the identity information of the person appearing in the home security video. For example, the target person identity can include children, the elderly, fathers, mothers, strangers, food delivery drivers, couriers, etc.

[0107] Target person behaviors refer to the actions or behaviors performed by the person appearing in the home security video. For example, target person behaviors can include but are not limited to walking, sitting, opening doors, and picking up objects.

[0108] In some embodiments, performing person identification processing on the home security video to obtain the identity of the target person may specifically include at least one of the following:

[0109] Capture the target person's face image from the home security video, perform face detection on the face image to obtain the target person's identity;

[0110] Extract the target person's voice data from the home security video, and perform voice recognition processing on the voice data to obtain the target person's identity;

[0111] The target person's body image is captured from the home security video, and the body feature recognition processing is performed on the body image to obtain the target person's identity.

[0112] It is understood that in the above-mentioned process of performing face detection processing on a facial image to obtain the identity of the target person, the facial image can be matched with a preset facial image to obtain a face matching result. If the face matching result indicates that a first facial image matching the facial image exists in the preset facial image, the identity of the person in the first facial image is used as the identity of the target person. If the face matching result indicates that a facial image matching the facial image does not exist in the preset facial image, the identity of the target person can be obtained by performing voice recognition processing on the voice data and / or performing body feature recognition processing on the body image. The preset facial image can be a facial image of a person with known identity information uploaded to the server in advance by the user.

[0113] It is understood that in the above-mentioned process of performing speech recognition processing on the voice data to obtain the target person's identity, the voice data can first be processed into speech-to-text to obtain the target text, and then the target person's identity can be identified based on the target text. For example, if a visitor outside the door utters the voice "Hello, a delivery has arrived", the visitor can be identified as a courier; for another example, if a visitor outside the door utters the voice "Hello, the takeaway has arrived", the visitor can be identified as a takeaway deliveryman.

[0114] It is understood that the aforementioned body image refers to an image containing a person's clothing. The identification of the target person by performing body feature recognition processing on the body image can specifically involve identifying the clothing style characteristics of the person in the body image and determining the target person's identity based on the clothing style characteristics. For example, by identifying the specific clothing style characteristics of a courier or food delivery driver, the courier or food delivery driver's identity information can be identified.

[0115] In some embodiments, the step of performing character behavior recognition processing on a home security video to obtain a target character's behavior can be implemented by performing character tracking processing on the target character in each video frame of the home security video to obtain position change data of the target character's key points, performing body posture detection processing based on the position change data of the key points to obtain a reference character behavior; extracting the target character's voice data from the home security video, performing speech-to-text processing on the voice data to obtain the character's speech; and determining the target character's behavior based on the reference character behavior and the character's speech. In this way, by jointly determining the target character's behavior based on the reference character behavior recognized based on the position change of the key points of the body and the character's speech recognized based on the voice data, the accuracy of the target character's behavior detection can be guaranteed. For example, if the target character is a visitor, the reference character behavior obtained by detecting the change of the visitor's key points in the home security video is "knocking on the door", and the character speech recognized by the visitor's voice data in the home security video is "Is anyone home?", the target character behavior can be determined as "someone knocking on the door" based on the "knocking on the door" and "Is anyone home?"

[0116] S203: Determine event description information of at least one event based on the target person's identity and the target person's behavior.

[0117] In some embodiments, each target person's behavior is identified as an event, and the time at which the event corresponding to the target person's behavior occurs is obtained. Event description information is generated based on the target person's behavior, the time at which the event corresponding to the target person's behavior occurs, and the identity of the target person who caused the target person's behavior. For example, an event description information may be "At 10:35 AM, the courier left the package at the door, and at 10:36 AM, the courier left."

[0118] S204: performing event screening processing on the event description information based on the event processing macro model to obtain target event description information, and performing event aggregation processing on the target event description information to obtain important event report information.

[0119] Target event description information refers to the descriptions of selected important events selected from the event description information. Important event report information refers to the report information generated by summarizing the important events in the event description information. The difference between target event description information and important event report information is that target event description information describes events in chronological order, while important event report information summarizes events involving the same person and / or location.

[0120] In some embodiments, executing step S204 may specifically include the following steps:

[0121] A1: Determine the reporting priority of at least one event, and generate an event handling prompt based on the reporting priority and event description information;

[0122] A2: Based on the event processing prompt word, the event processing large model is used to perform event screening processing on the event description information to obtain the target event description information, and the target event description information is then summarized to obtain important event report information.

[0123] In step A1, the reporting priority can be used to characterize whether an event is an important event. The reporting priority can include high priority and low priority. When the reporting priority of an event is high priority, the event is determined to be an important event. When the reporting priority of an event is low priority, the event is determined to be not an important event.

[0124] Specifically, the execution of determining the reporting priority of at least one event may include: obtaining preset event information corresponding to the user, and determining the first reporting priority of at least one event based on the preset event information; and / or, obtaining the user's historical video access behavior, determining the user's preferred event information based on the historical video access behavior, and determining the second reporting priority of at least one event based on the preferred event information.

[0125] It is understandable that the preset event information of interest may be information characterizing the event type or person identity that the user is concerned about, and the preset event information of interest may be information that is pre-set by the user on the user terminal and uploaded to the server by the user terminal. In the process of determining the first reporting priority of at least one event, the event type of interest and the person identity of interest may be determined from the preset event information of interest, a first event belonging to the event type of interest may be determined in at least one event, a second event corresponding to the person identity of interest may be determined in at least one event, the first reporting priority of the first event and the second event may be determined as a high priority, and the first reporting priority of the remaining events in at least one event except the first event and the second event may be determined as a low priority.

[0126] It is understandable that historical video access behavior may include the behavior of users accessing videos of specific events, the behavior of users marking videos of specific events, the behavior of users downloading videos of specific events, etc., and the preferred event information corresponding to the user can be determined based on the frequency of historical access to videos of specific events and / or the frequency of users marking videos of specific events. For example, the identity of the character in the video with a higher access frequency is determined as the preferred character identity corresponding to the user, and the event type to which the specific event in the video with a higher access frequency belongs is determined as the preferred event type corresponding to the user, and preferred event information including the preferred character identity and the preferred event type can be generated. In the process of determining the second reporting priority of at least one event, the preferred event type and preferred character identity corresponding to the user can be determined from the preferred event information, a third event belonging to the preferred event type can be determined in at least one event, a fourth event corresponding to the preferred character identity can be determined in at least one event, the second reporting priority of the third event and the fourth event can be determined as a high priority, and the second reporting priority of the remaining events in at least one event except the third event and the fourth event can be determined as a low priority.

[0127] In the process of determining the first reporting priority of at least one event based on preset event information, and determining the second reporting priority of at least one event based on preference event information, if there is at least one high priority between the first reporting priority and the second reporting priority of the target event, the reporting priority of the target event is determined to be a high priority; if the first reporting priority and the second reporting priority of the target event are both low priorities, the reporting priority of the target event is determined to be a low priority.

[0128] In step A1, the event processing prompt word may refer to task description information used to instruct the event processing large model to perform event screening and event aggregation. Specifically, the event processing prompt word may be generated including output instructions, event description information, and the reporting priority of at least one event. The output instructions may include style description information used to instruct the event processing large model to output important event reporting information in the language style. For example, the style description information may instruct the event processing large model to describe important events in a human speaking style.

[0129] In step A2, the event processing prompt word can be input into the event processing large model. The event processing large model can perform event screening processing on the event description information according to the reporting priority of at least one event to obtain the target event description information, such as retaining the description information of events with a high reporting priority in the event description information, and deleting the description information of events with a low reporting priority from the event description information to obtain the target event description information. Furthermore, the event processing large model aggregates the events in the target event description information according to the same person identity and / or the same location, and obtains coherent event description information of the same person identity and / or coherent event description information of the same location, and important event reporting information based on the coherent event description information of the same person identity and / or the coherent event description information of the same location. For example, an important event reporting information can be "Good afternoon, master, everything is normal at home today! The baby slept from 3 to 5 o'clock, the dog also ate, and there are 2 express deliveries at the door."

[0130] In this way, through the powerful text processing capabilities of the large model, the descriptive information of important events can be quickly reported and summarized, so that users can quickly understand the important events that occurred within the preset time period based on the important event report information, which increases the convenience for users to view information they are interested in or important information.

[0131] S205 , obtaining a target video clip corresponding to the important event report information, generating a video jump link based on the target video clip, and performing data splicing processing on the important event report information and the video jump link to obtain comprehensive important event report information.

[0132] In some embodiments, a target video clip corresponding to an important event in the important event report information can be extracted from the home security video. Furthermore, a video jump link associated with the target video clip can be generated. The video jump link is used to jump to and display the target video clip. Video jump text corresponding to the video jump link is generated. The important event report information and the video jump text are then merged to obtain comprehensive important event report information. It is understood that when a user clicks the video jump text, the target video clip is displayed via the video jump link, allowing the user to quickly view the video clip of the important event.

[0133] For example, following the example of the important event reporting information in step S204, a comprehensive report information of an important event can be "Good afternoon, master, everything is fine at home today! The baby slept from 3 to 5 o'clock, the dog also ate, there are 2 express deliveries at the door, click to view", where "click to view" is the jump text.

[0134] S206: Send the important event comprehensive report information to the terminal, so that the terminal displays the important event comprehensive report information on the smart home management interface.

[0135] It's understood that a smart home management interface refers to the user interface within smart home management software used to manage home hardware. Smart home management software enables convenient management of home hardware, anytime access to the home network, quick processing of home services, and sharing among family members.

[0136] Specifically, the server sends the important event comprehensive report information to the terminal, so that the terminal can instantly display the important event comprehensive report information to the user, so that the user can quickly view the information of the important events that occurred within a preset time period.

[0137] In an embodiment of the present application, a server obtains a home security video captured by at least one camera within a preset time period, performs person identity recognition processing on the home security video to obtain the identity of a target person, performs person behavior recognition processing on the home security video to obtain the behavior of a target person, determines event description information of at least one event based on the identity of the target person and the behavior of the target person, performs event screening processing on the event description information based on an event processing large model to obtain target event description information, and performs event aggregation processing on the target event description information to obtain important event report information, obtains a target video clip corresponding to the important event report information, generates a video jump link based on the target video clip, performs data splicing processing on the important event report information and the video jump link to obtain comprehensive report information of important events, and sends the comprehensive report information of important events to a terminal so that the terminal displays the comprehensive report information of important events on the smart home management interface. In this way, the server identifies the event description information corresponding to at least one event in the home security video, and then uses the powerful text generation capability of the large model to generate important event report information based on the event description information, and sends the important event comprehensive report information obtained by splicing the important event report information and the video jump link to the terminal, so that the terminal user can not only quickly understand the text information of important events that occurred in the home environment video collected by multiple devices, but also can quickly view the video of important events that occurred in the home environment video collected by multiple devices, without the user having to view multiple videos of multiple devices one by one to obtain information about important events that occurred in the home environment within a preset time period. Therefore, the embodiment of the present application increases the convenience of users viewing important event information and event videos in home environment videos, and also improves the efficiency of users viewing important event information and event videos.

[0138] See Figure 4 , Figure 4 This is a flow chart of another embodiment of a data processing method proposed in this application.

[0139] Specifically, the data processing method is executed by a terminal, and the method includes:

[0140] S301: Obtain important event report information sent by the server.

[0141] Specifically, the important event reporting information is obtained by the server performing event aggregation processing on the event description information of at least one event based on the event processing big model, and the event description information is obtained by the server performing event recognition processing based on the home security video collected by at least one camera.

[0142] For details on how the server obtains important event reporting information, see Figure 2 or Figure 3 The description of the relevant steps in the illustrated embodiment will not be repeated in detail here.

[0143] S302: Display important event report information on the smart home management interface.

[0144] It's understood that a smart home management interface refers to the user interface within smart home management software used to manage home hardware. Smart home management software enables convenient management of home hardware, anytime access to the home network, quick processing of home services, and sharing among family members.

[0145] Specifically, the terminal can display important event report information when the user opens the smart home management interface.

[0146] In an embodiment of the present application, the terminal obtains important event report information sent by the server and displays the important event report information on the smart home management interface. In this way, users can conveniently view text information about important events that occurred in home environment videos collected by multiple devices based on the important event report information, without having to view multiple videos from multiple devices one by one to obtain information about important events or events of interest. This increases the convenience of users viewing important event information or events of interest in home environment videos. In addition, the important event report information corresponding to each target scene is generated by the server, eliminating the need to consume the terminal's computing power resources for determination, thus saving the terminal's computing power resources.

[0147] See Figure 5 , Figure 5 This is a flow chart of another embodiment of a data processing method proposed in this application.

[0148] Specifically, the data processing method is executed by a terminal, and the method includes:

[0149] S401: Obtain important event report information sent by the server.

[0150] Specifically, the implementation of step S401 can be found in Figure 4 The description of the relevant steps in the illustrated embodiment will not be repeated in detail here.

[0151] S402: Determine an event reporting area in the smart home management interface.

[0152] S403: Display important event report information in the event report area.

[0153] In some embodiments, a display area at a preset location in the smart home management interface can be designated as an event reporting area. For example, the smart home management interface may include a search area and a display area, with the search area located above the display area, such as at the top of the interface. A display area at a preset location indicated by a preset coordinate area below the search area can be designated as the event reporting area. This allows users to quickly view text information about important events without having to flip through pages.

[0154] For example, see Figure 6 A schematic diagram showing an important event reporting information is shown. Figure 6 The text shown in the figure, "Good afternoon, owner. Everything is normal at home today! The baby slept from 3 to 5 o'clock, the dog also ate, and there are 2 express deliveries at the door" is the important event report information. The area where the text is located in the interface is the event report area. In this way, the important event report information is displayed to the user through the event report area near the top of the interface, so that the user can quickly view the important event report information without turning pages.

[0155] Optionally, the embodiment of the present application also performs: in response to the event information setting operation input by the user, displaying a preset event information configuration interface; obtaining the preset event information of interest input by the user for the preset event information configuration interface, and sending the preset event information of interest to the server, the preset event information of interest is used by the server to generate event processing prompt words based on the event description information, and based on the event processing prompt words, use the event processing big model to perform event screening processing on the event description information to obtain target event description information, and perform event summary processing on the target event description information to obtain important event report information.

[0156] Optionally, the embodiment of the present application may further perform the following steps: obtaining important event comprehensive report information sent by the server, the important event comprehensive report information being obtained by the server through data splicing processing based on the important event report information and a video jump link, the video jump link being generated by the server based on a target video clip corresponding to the important event report information; and displaying the important event comprehensive report information on the smart home management interface. In this way, the terminal user can quickly view not only the text information of the important event, but also the video of the important event.

[0157] In the data processing method provided in the embodiment of the present application, the terminal obtains important event reporting information sent by the server, determines the event reporting area in the smart home management interface, and displays the important event reporting information in the event reporting area. In this way, the user can conveniently view the text information of important events or events of concern that occur in the home environment video collected by multiple devices in the smart home management interface without turning pages, and the user does not need to view multiple videos of multiple devices one by one to obtain information about important events or events of concern, which increases the convenience of users in viewing important event information or event information in the home environment video. In addition, the important event reporting information corresponding to each target scene is generated by the server, and there is no need to consume the computing power resources of the terminal for determination, thereby saving the computing power resources of the terminal.

[0158] The following will be combined Figure 7 , the data processing device provided in the embodiment of the present application is introduced in detail. It should be noted that, Figure 7 The data processing device shown is used to execute the application Figures 2 and 3 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figures 2 and 3 The embodiment shown.

[0159] See Figure 7 , which shows a schematic diagram of the structure of a data processing device according to an embodiment of the present application. The data processing device 1 can be implemented as all or part of the device through software, hardware, or a combination of both. According to some embodiments, the data processing device 1 includes a video acquisition module 11, an event processing module 12, and an information sending module 13, which are specifically used to:

[0160] The video acquisition module 11 is configured to acquire home security videos captured by at least one camera within a preset time period, and perform event recognition processing on the home security videos to obtain event description information of at least one event;

[0161] An event processing module 12 is configured to aggregate the event description information based on the event processing model to obtain important event report information;

[0162] The information sending module 13 is used to send the important event report information to the terminal, so that the terminal displays the important event report information on the smart home management interface.

[0163] Optionally, the event processing module 12 includes:

[0164] The event processing unit is used to perform event screening processing on the event description information based on the event processing model to obtain target event description information, and to perform event aggregation processing on the target event description information to obtain important event report information.

[0165] Optionally, the event processing unit includes:

[0166] a first event processing subunit, configured to determine a reporting priority of the at least one event, and generate an event processing prompt word based on the reporting priority and the event description information;

[0167] The second event processing sub-unit is used to perform event screening processing on the event description information based on the event processing prompt word using the event processing large model to obtain target event description information, and to perform event aggregation processing on the target event description information to obtain important event report information.

[0168] Optionally, the first event processing subunit includes:

[0169] A first priority determination subunit is configured to obtain preset event information corresponding to a user, and determine a first reporting priority of the at least one event based on the preset event information; and / or

[0170] The second priority determination subunit is used to obtain the user's historical video access behavior, determine the preference event information corresponding to the user based on the historical video access behavior, and determine the second reporting priority of the at least one event based on the preference event information.

[0171] Optionally, the data processing device 1 further includes:

[0172] An event splicing module is used to obtain a target video clip corresponding to the important event report information, generate a video jump link based on the target video clip, and perform data splicing processing on the important event report information and the video jump link to obtain comprehensive important event report information;

[0173] The information sending module is used to send the important event comprehensive report information to the terminal, so that the terminal displays the important event comprehensive report information on the smart home management interface.

[0174] Optionally, the video acquisition module 11 includes:

[0175] a video recognition unit, configured to perform person identification processing on the home security video to obtain the identity of a target person, and perform person behavior identification processing on the home security video to obtain the behavior of the target person;

[0176] An event generating unit is configured to determine event description information of at least one event based on the target person's identity and the target person's behavior.

[0177] The following will be combined Figure 8, the data processing device provided in the embodiment of the present application is introduced in detail. It should be noted that, Figure 8 The data processing device shown is used to execute the application Figures 4 to 6 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figures 2 to 6 The embodiment shown.

[0178] See Figure 8 , which shows a schematic diagram of the structure of the data processing device of an embodiment of the present application. The data processing device 2 can be implemented as all or part of the device through software, hardware, or a combination of both. According to some embodiments, the data processing device 1 includes an information acquisition module 21 and an event display module 22, which are specifically used to:

[0179] An information acquisition module 21 is configured to acquire important event report information sent by a server, wherein the important event report information is obtained by the server performing event aggregation processing on event description information of at least one event based on an event processing macro model, and the event description information is obtained by the server performing event recognition processing based on home security video captured by at least one camera;

[0180] The event display module 22 is used to display the important event report information on the smart home management interface.

[0181] Optionally, the event display module 22 includes:

[0182] A first display processing unit is used to determine an event reporting area in the smart home management interface;

[0183] The second display processing unit is configured to display the important event report information in the event report area.

[0184] Optionally, the data processing device 2 further includes:

[0185] An event configuration module, configured to display a preset event information configuration interface in response to an event information setting operation input by a user;

[0186] A data sending module is used to obtain the preset event information of interest input by the user for the preset event information configuration interface, and send the preset event information of interest to the server. The preset event information of interest is used by the server to generate event processing prompt words based on the event description information, and to use the event processing big model to perform event screening processing on the event description information based on the event processing prompt words to obtain target event description information, and to perform event summary processing on the target event description information to obtain important event report information.

[0187] Please refer to Figure 9 , which shows a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. Specifically, the electronic device may be the server or terminal described in the above embodiments. The electronic device in the embodiment of the present application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0188] The processor 110 may include one or more processing cores. The processor 110 utilizes various interfaces and circuits to connect various components within the electronic device. It executes instructions, programs, code sets, or instruction sets stored in the memory 120, as well as accesses data stored in the memory 120, to perform various functions of the electronic device and process data. Optionally, the processor 110 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 110 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 110 and may be implemented separately via a communications chip.

[0189] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an iOS system developed by Apple, including a system deeply developed based on the iOS system, or other systems.

[0190] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0191] The input device 130 is used to receive input commands or data and includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch-sensitive device. The output device 140 is used to output commands or data and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 may be combined, and the input device 130 and the output device 140 may be a touch-sensitive display.

[0192] The touch display screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

[0193] In addition, those skilled in the art will appreciate that the structures of the smart home devices shown in the above figures do not limit the smart home devices. Smart home devices may include more or fewer components than shown, or may combine certain components or arrange the components differently. For example, smart home devices may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WiFi) modules, power supplies, Bluetooth modules, and other components, which will not be described in detail here.

[0194] In some embodiments, Figure 9The processor 110 in the electronic device shown may be used to call the program of the data processing method stored in the memory 120 and specifically perform the following operations:

[0195] Acquiring a home security video captured by at least one camera within a preset time period, and performing event recognition processing on the home security video to obtain event description information of at least one event;

[0196] Based on the event processing model, the event description information is aggregated to obtain important event report information;

[0197] The important event report information is sent to the terminal, so that the terminal displays the important event report information on the smart home management interface.

[0198] In other embodiments, Figure 9 The processor 110 in the electronic device shown may be used to call a program of another data processing method stored in the memory 120 and specifically perform the following operations:

[0199] Obtaining important event report information sent by a server, the important event report information being obtained by the server performing event aggregation processing on event description information of at least one event based on an event processing macromodel, the event description information being obtained by the server performing event recognition processing based on home security video captured by at least one camera;

[0200] The important event report information is displayed on the smart home management interface.

[0201] An embodiment of the present application further provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the data processing method described in the above embodiments.

[0202] An embodiment of the present application further provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the data processing method described in the above embodiments.

[0203] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0204] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: Applied to a server, the method includes: Acquiring a home security video captured by at least one camera within a preset time period, and performing event recognition processing on the home security video to obtain event description information of at least one event; Based on the event processing model, the event description information is aggregated to obtain important event report information; The important event report information is sent to the terminal, so that the terminal displays the important event report information on the smart home management interface.

2. The method according to claim 1, characterized in that The event processing macro model is used to aggregate the event description information to obtain important event report information, including: Based on the event processing macro model, event screening processing is performed on the event description information to obtain target event description information, and event aggregation processing is performed on the target event description information to obtain important event report information.

3. The method according to claim 2, characterized in that The performing event screening processing on the event description information based on the event processing macro model to obtain target event description information, and performing event aggregation processing on the target event description information to obtain important event report information, includes: Determining a reporting priority of the at least one event, and generating an event processing prompt word based on the reporting priority and the event description information; Based on the event processing prompt word, the event processing large model is used to perform event screening processing on the event description information to obtain target event description information, and event aggregation processing is performed on the target event description information to obtain important event report information.

4. The method according to claim 3, characterized in that Determining the reporting priority of the at least one event includes: Obtaining preset event information corresponding to the user, and determining a first reporting priority of the at least one event based on the preset event information; and / or, Acquire historical video access behaviors of a user, determine preferred event information corresponding to the user based on the historical video access behaviors, and determine a second reporting priority of the at least one event based on the preferred event information.

5. The method according to claim 1, wherein Before sending the important event report information to the terminal, the method further includes: Obtaining a target video clip corresponding to the important event report information, generating a video jump link based on the target video clip, and performing data splicing processing on the important event report information and the video jump link to obtain comprehensive important event report information; The sending of the important event report information to the terminal so that the terminal displays the important event report information on the smart home management interface includes: The important event comprehensive report information is sent to the terminal, so that the terminal displays the important event comprehensive report information on the smart home management interface.

6. A data processing method, characterized in that: Applied to a terminal, the method includes: Obtaining important event report information sent by a server, the important event report information being obtained by the server performing event aggregation processing on event description information of at least one event based on an event processing macromodel, the event description information being obtained by the server performing event recognition processing based on home security video captured by at least one camera; The important event report information is displayed on the smart home management interface.

7. A data processing device, characterized in that: Applied to a server, the device includes: a video acquisition module, configured to acquire home security videos captured by at least one camera within a preset time period, and perform event recognition processing on the home security videos to obtain event description information of at least one event; An event processing module is used to perform event aggregation processing on the event description information based on the event processing model to obtain important event report information; The information sending module is used to send the important event report information to the terminal, so that the terminal displays the important event report information on the smart home management interface.

8. A data processing device, characterized in that: Applied to a terminal, the device includes: an information acquisition module, configured to acquire important event report information sent by a server, the important event report information being obtained by the server performing event aggregation processing on event description information of at least one event based on an event processing macromodel, the event description information being obtained by the server performing event recognition processing based on home security video captured by at least one camera; The event display module is used to display the important event report information on the smart home management interface.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 5 or 6.

10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method according to any one of claims 1 to 5 or 6.

Images