A video processing method, device, storage medium and electronic device

Abstract

Embodiments of the present application disclose a video processing method and device, a storage medium and an electronic device, wherein the method comprises: a smart home device collects security video of an environment in which the smart home device is located, performs basic event identification on the security video by using a basic event identification model to obtain event video cover information, and sends the event video cover information and the security video to a cloud service platform; the cloud service platform performs event scene analysis on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on the security video based on the event scene classification information and the event video cover information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a video processing method and device, a storage medium, and an electronic device. BACKGROUND

[0002] With the development of the smart home industry, smartness is increasingly favored by the market and users, and safe and convenient use is a basic goal pursued by the market and users. Smart home devices such as smart doorbells, smart cameras, and smart door locks can collect videos under the environment and synchronize the videos to the cloud, so that users can view the videos by accessing the cloud. SUMMARY

[0003] The present application provides a video processing method and device, a storage medium, and an electronic device. The technical solution is as follows:

[0004] In a first aspect, the present application provides a video processing method applied to a smart home device, the method comprising:

[0005] collecting a security video of an environment;

[0006] performing basic event recognition on the security video using a basic event recognition model to obtain event video cover information;

[0007] sending the event video cover information and the security video to a cloud service platform, so that the cloud service platform performs event scene analysis based on the event video cover information using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on the security video based on the event scene classification information and the event video cover information.

[0008] In a feasible implementation, the performing basic event recognition on the security video using a basic event recognition model to obtain a basic event recognition result and event video cover includes:

[0009] inputting the security video into a basic event recognition model;

[0010] performing basic scene object detection on the security video through the basic event recognition model to obtain basic scene object image information, performing basic event recognition based on the basic scene object image information to obtain a basic event recognition result, and determining a basic event video indicated by the basic event recognition result from the security video;

[0011] performing video cover screening processing on the basic event video through the basic event recognition model to obtain an event video cover, and outputting event video cover information;

[0012] The event video cover information includes an event video cover image and video cover meta information corresponding to the event video cover image.

[0013] In an implementation, the event video cover is obtained by performing video cover screening on the basic event video based on the basic event recognition model, including:

[0014] The video frame scene information of the basic event video frame included in the basic event video is extracted based on the basic event recognition model, the basic event video frame is scored based on the video frame scene information, and at least one event video cover frame is determined from the basic event video frame based on the basic scene score.

[0015] In a second aspect, the embodiments of the present application provide a video processing method applied to a cloud service platform, including:

[0016] Receiving event video cover information and security video uploaded by a smart home device, the event video cover information being information obtained by performing basic event recognition on the security video by the smart home device based on a basic event recognition model;

[0017] Performing event scene analysis based on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and performing video scene classification processing on the security video based on the event scene classification information and the event video cover information.

[0018] In an implementation, the event scene analysis based on the event video cover information by using an event scene analysis large model to obtain event scene classification information, including:

[0019] Obtaining an event video cover image and video cover meta information corresponding to the event video cover image from the event video cover information;

[0020] Constructing an event scene analysis prompt word for the event video cover image based on the video cover meta information;

[0021] Inputting the event scene analysis prompt word and the event video cover image into an event scene analysis large model to perform event scene analysis, and outputting event scene classification information.

[0022] In an implementation, the event scene analysis prompt word for the event video cover image is constructed based on the video cover meta information, including:

[0023] Adding the video cover meta information to an event scene analysis template to obtain the event scene analysis prompt word for the event video cover image.

[0024] In an implementable embodiment, the event scene analysis prompt word and the event video cover image are input into an event scene analysis large model for event scene analysis, and event scene classification information is output.

[0025] The event scene analysis prompt word and the event video cover image are input into an event scene analysis large model.

[0026] The event scene analysis large model performs image-to-text conversion on the event video cover image based on the event scene analysis prompt word to obtain event cover description information, performs key semantic analysis on the event cover description information to obtain key semantic information, and performs event scene classification on the key semantic information to obtain event scene classification information.

[0027] The event scene classification information is output by the scene analysis large model.

[0028] In an implementable embodiment, the method further comprises:

[0029] An initial event scene analysis large model for event scene analysis is created using a basic large language model.

[0030] Sample event video cover information is obtained, and event scene classification information labels are labeled on the sample event video cover information.

[0031] The initial event scene analysis large model is trained at least once using the sample event video cover information. During the model training process, the initial event scene analysis large model is used to perform event scene analysis processing on the sample event video cover information to obtain predicted event scene classification information. The initial event scene analysis large model is adjusted in model parameters based on the predicted event scene classification information and the event scene classification information labels to obtain an event scene analysis large model after model training.

[0032] In an implementable embodiment, the video scene classification processing of the security video based on the event scene classification information and the event video cover information comprises:

[0033] An event video cover image in the event video cover information and video cover meta information corresponding to the event video cover image are determined.

[0034] An event scene video is extracted from the security video based on the video cover meta information, and the event video cover image is used as a video cover of the event scene video.

[0035] The event scene video is associated with an event scene classification set corresponding to the event scene classification information.

[0036] In a third aspect, the embodiments of the present application provide a video processing device applied to a smart home device, the device comprising:

[0037] a video collection module configured to collect a security video of an environment;

[0038] an event identification module configured to perform basic event identification on the security video by using a basic event identification model to obtain event video cover information;

[0039] a scene analysis module configured to send the event video cover information and the security video to a cloud service platform, so that the cloud service platform performs event scene analysis by using an event scene analysis large model based on the event video cover information to obtain event scene classification information, and performs video scene classification processing on the security video based on the event scene classification information and the event video cover information.

[0040] In a feasible implementation, the event identification module is configured to:

[0041] input the security video into the basic event identification model;

[0042] perform basic scene object detection on the security video by using the basic event identification model to obtain basic scene object image information, perform basic event identification based on the basic scene object image information to obtain a basic event identification result, and determine a basic event video indicated by the basic event identification result from the security video;

[0043] perform video cover screening processing on the basic event video by using the basic event identification model to obtain an event video cover, and output event video cover information;

[0044] The event video cover information comprises an event video cover image and video cover meta information corresponding to the event video cover image.

[0045] In a feasible implementation, the event identification module is configured to:

[0046] extract video frame scene information of a basic event video frame contained in the basic event video by using the basic event identification model, perform basic scene scoring on the basic event video frame based on the video frame scene information, and determine at least one frame of event video cover from the basic event video frame based on the basic scene scoring.

[0047] In a fourth aspect, the embodiments of the present application provide a video processing device applied to a cloud service platform, the method comprising:

[0048] The data receiving module is configured to receive event video cover information uploaded by the smart home device and security video, wherein the event video cover information is information obtained by performing basic event identification on the security video by the smart home device using a basic event identification model;

[0049] The scene analysis module is configured to perform event scene analysis on the event video cover information using an event scene analysis large model to obtain event scene classification information, and perform video scene classification processing on the security video based on the event scene classification information and the event video cover information.

[0050] In an implementable embodiment, the scene analysis module is configured to:

[0051] obtain an event video cover image and video cover meta information corresponding to the event video cover image from the event video cover information;

[0052] construct an event scene analysis prompt word for the event video cover image based on the video cover meta information;

[0053] input the event scene analysis prompt word and the event video cover image into an event scene analysis large model to perform event scene analysis, and output event scene classification information.

[0054] In an implementable embodiment, the scene analysis module is configured to:

[0055] add the video cover meta information to an event scene analysis template to obtain an event scene analysis prompt word for the event video cover image.

[0056] In an implementable embodiment, the scene analysis module is configured to:

[0057] input the event scene analysis prompt word and the event video cover image into an event scene analysis large model;

[0058] perform image-to-text conversion on the event video cover image based on the event scene analysis prompt word by the event scene analysis large model to obtain event cover description information, perform key semantic analysis on the event cover description information to obtain key semantic information, and perform event scene classification on the key semantic information to obtain event scene classification information;

[0059] output the event scene classification information by the scene analysis large model.

[0060] In an implementable embodiment, the apparatus is further configured to:

[0061] create an initial event scene analysis large model for event scene analysis of a scene using a basic large language model.

[0062] Obtaining sample event video cover information, and labeling event scene classification information tags on the sample event video cover information.

[0063] At least one round of model training is performed on the initial event scene analysis large model by using the sample event video cover information. In the model training process, the initial event scene analysis large model is used to perform event scene analysis processing on the sample event video cover information to obtain predicted event scene classification information. The initial event scene analysis large model is adjusted in model parameters based on the predicted event scene classification information and the event scene classification information tags to obtain the event scene analysis large model after model training.

[0064] In a possible implementation, the scene analysis module is configured to:

[0065] Determine event video cover images in the event video cover information and video cover meta information corresponding to the event video cover images;

[0066] Extract event scene videos from the security video based on the video cover meta information, and use the event video cover images as video covers of the event scene videos.

[0067] Associate the event scene videos with an event scene classification set corresponding to the event scene classification information.

[0068] In a fifth aspect, an embodiment of the present application provides a computer storage medium, which stores a plurality of instructions. The instructions are adapted to be loaded by a processor and perform the method steps described above.

[0069] In a sixth aspect, an embodiment of the present application provides an electronic device, which can include a processor and a memory. The memory stores a computer program, which is adapted to be loaded by the processor and perform the method steps described above.

[0070] The technical solutions provided by some embodiments of the present application have at least the following beneficial effects:

[0071] In one or more embodiments of the present application, the smart home device collects security videos of the environment, uses a basic event recognition model to perform basic event recognition on the security videos to obtain event video cover information, sends the event video cover information and the security videos to a cloud service platform, the auxiliary service platform uses an event scene analysis large model to perform event scene analysis based on the event video cover information to obtain event scene classification information, and performs video scene classification processing on the security videos based on the event scene classification information and the event video cover information. After the video scene is accurately classified based on the event video cover and the event scene classification information, the user can access the cloud service platform, quickly query the interested video based on the video classification, and review the video, thereby improving the convenience and efficiency of video review of the user, and strengthening the security protection of the family, making the smart home device application more perfect and practical. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0073] Figure 1 is a flowchart of a video processing method provided by an embodiment of the present application;

[0074] Figure 2 is a flowchart of another embodiment of a video processing method provided by an embodiment of the present application;

[0075] Figure 3 is a flowchart of a video processing method provided by an embodiment of the present application;

[0076] Figure 4 is a flowchart of another embodiment of a video processing method provided by an embodiment of the present application;

[0077] Figure 5 is an interface diagram of an intelligent cover search provided by an embodiment of the present application;

[0078] Figure 6 is a scene diagram of an intelligent cover search provided by an embodiment of the present application;

[0079] Figure 7 is a structural diagram of a video processing device provided by an embodiment of the present application;

[0080] Figure 8 is a structural diagram of another video processing device provided by an embodiment of the present application;

[0081] Figure 9 is a structural schematic diagram of an electronic device provided by an embodiment of the present application.

[0082] Figure 10 is a structural schematic diagram of an operating system and a user space provided by an embodiment of the present application.

[0083] Figure 11 is Figure 10 an architecture diagram of an Android operating system in the present application.

[0084] Figure 12 is Figure 10 an architecture diagram of an IOS operating system in the present application. DETAILED DESCRIPTION

[0085] The technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0086] In the description of the present application, it should be understood that the terms "first", "second" and the like are only used for the purpose of description and should not be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units not listed, or optionally includes other steps or units inherent to the process, method, product or device. Those of ordinary skill in the art can understand the specific meaning of the above terms in the present application according to the specific circumstances. In addition, in the description of the present application, "multiple" means two or more, unless otherwise specified. "And / or" describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents a "or" relationship between the associated objects.

[0087] In the related art, the smart home device usually uploads only the collected video to the cloud service platform, and the video saved by the cloud service platform is calculated in hours, and the number of videos accumulated in a period of time is relatively large. If the user intends to review the previous scene video, the video of the interested time period cannot be quickly and conveniently found. Based on this, it can be seen that the video processing in the related art causes the limitation that the user video review is not convenient.

[0088] The present application will be described in detail below with reference to specific embodiments.

[0089] In one embodiment, as shown in Figure 1 A video processing method is proposed, which can be implemented by a computer program and run on a video processing device based on the Von Neumann architecture. The computer program can be integrated in an application or run as a standalone tool application. The video processing device can be a smart home device, including but not limited to a smart door lock, a smart doorbell, a smart camera, a smart home device, etc.

[0090] Specifically, the video processing method includes:

[0091] S102: Collecting security videos of the environment;

[0092] The video acquisition function of the smart home device is usually realized by an integrated camera. The smart home device can start the video acquisition function when it identifies that there is a monitoring object (such as a person, an animal, or a movable object) in the environment, and collect security videos of the environment;

[0093] Specifically, the smart home device is configured with a camera that can record high-definition videos. The camera can support night vision function and use infrared or other low-light technology to record clear videos at night or in insufficient light environment.

[0094] For example, the smart home device collects security videos of the environment in real time or periodically;

[0095] Further, the smart home device is configured with a motion detection mode. In the motion detection mode, when the smart home device detects that an object in the environment moves, the smart home device automatically starts video recording to collect a video clip. The collected video clip can be referred to as a security video.

[0096] S104: Using a basic event recognition model to perform basic event recognition on the security video to obtain event video cover information;

[0097] Illustratively, the cloud service platform can create and train a basic event recognition model based on a machine learning model in advance, and deploy the basic event recognition model on the smart home device. In actual application, the smart home device inputs the collected security video into the basic event recognition model, and performs basic event recognition on the security video by the basic event recognition model to obtain event video cover information.

[0098] The basic event can be one or more basic scene object events counted by a cloud service platform through big data, such as a courier arriving at the door, family members coming home, a property manager arriving at the door, crying, a pet appearing, and a user watching TV in a home life scene. The preset basic event is used to train the basic event recognition model, so that the event video cover information can be obtained by recognizing the basic event from the security video with the help of the basic event recognition model.

[0099] The event video cover information can include an event video cover image, and in some embodiments, can also include video cover meta information of the event video cover image.

[0100] The video cover meta information is information describing the video cover image in the security video, such as a time point corresponding to the video cover image, a video time period of the video cover image corresponding to the basic event, device information of the video cover image, a home location corresponding to the cover, an event type, and the like.

[0101] For example, the following illustrates a model training process of the basic event recognition model as follows:

[0102] The basic event recognition model is obtained by model training of a machine learning model based on security video samples with labeled event video cover information tags. The security video samples include sample system running state topology and sample target device running information.

[0103] The following interprets the model training process of the basic event recognition model as follows:

[0104] Model creation: create an initial basic event recognition model for the basic event recognition scene based on a machine learning model;

[0105] Sample data acquisition: acquire a large number of security video samples.

[0106] Sample data labeling: introduce expert end service to manually label the corresponding event video cover information tags of the security video samples based on the requirements of the basic event recognition scene. The event video cover information tags include event video cover image tags and video cover meta information tags.

[0107] Model training process: input the sample data into the initial basic event recognition model for at least one round of model training to obtain predicted event video cover information. Based on the predicted event video cover information and the event video cover information tags, a model loss function is used to determine a model loss value. Based on the model loss value, the initial basic event recognition model is adjusted for model parameters until a model training end condition is met to obtain the basic event recognition model.

[0108] Optionally, the model loss function can be a hinge loss function, a cross-entropy loss function, a feature distance loss function, etc.

[0109] Optionally, the model end training condition of the model can include, for example, a value of the loss function being less than or equal to a preset loss function threshold, a number of iterations reaching a preset number threshold, etc. The specific model end training condition can be determined based on actual conditions, and is not limited here.

[0110] It should be noted that the machine learning model involved in one or more embodiments of the present specification includes, but is not limited to, fitting of one or more of a convolutional neural network (CNN) model, a deep neural network (DNN) model, a recurrent neural network (RNN), an embedding model, a gradient boosting decision tree (GBDT) model, a logistic regression (LR) model, etc.

[0111] In a feasible implementation, the security video can be input to a basic event recognition model, a basic scene object image information is obtained by performing basic scene object detection on the security video by the basic event recognition model, a basic event recognition result is obtained based on the basic scene object image information, a basic event video indicated by the basic event recognition result is determined from the security video, an event video cover is obtained by performing video cover screening processing on the basic event video by the basic event recognition model, and event video cover information is output.

[0112] The event video cover information includes an event video cover image and video cover meta information corresponding to the event video cover image.

[0113] S106: The event video cover information and the security video are sent to a cloud service platform, so that the cloud service platform performs event scene analysis based on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on the security video based on the event scene classification information and the event video cover information.

[0114] It can be understood that the event video cover information and the security video can be sent to the service platform at the same time, or the event video cover information and the security video can be sent to the service platform at different time points.

[0115] The cloud service platform and at least one smart home device in the smart home system establish a communication connection through a network, and based on the communication connection, data interaction in a video processing process is completed. The smart home device sends event video cover information and security video to the cloud service platform. The cloud service platform performs event scene analysis based on the event video cover information using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on the security video based on the event scene classification information and the event video cover information. A user can access the cloud service platform (such as through a terminal) to quickly query interesting videos according to video classification for video review, which can facilitate the user to review more specific scene videos.

[0116] In one or more embodiments of the specification, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data (such as the target compressed package) exchanged through the network. In addition, all or some links can be encrypted using conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. In other embodiments, custom and / or dedicated data communication technologies can be used instead of or in addition to the above data communication technologies.

[0117] In one or more embodiments of the present application, the smart home device collects security videos of the environment, uses a basic event recognition model to perform basic event recognition on the security videos to obtain event video cover information, sends the event video cover information and the security videos to a cloud service platform, assists the service platform in performing event scene analysis based on the event video cover information using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on the security videos based on the event scene classification information and the event video cover information. After the video scene is accurately classified based on the event video cover and the event scene classification information, the user can access the cloud service platform, quickly query interesting videos based on video classification for video review, thereby improving the convenience and efficiency of video review, and also enhancing the security protection of the family, making the smart home device application more perfect and practical.

[0118] Please refer to Figure 2 , Figure 2 is a flowchart of another embodiment of a video processing method proposed in the present application. Specifically:

[0119] S202: Collecting security videos of the environment;

[0120] For details, refer to the method steps of other embodiments of the present application, which will not be repeated here.

[0121] S204: Inputting the security videos into a basic event recognition model;

[0122] S206: Performing basic scene object detection on the security videos by the basic event recognition model to obtain basic scene object image information, performing basic event recognition based on the basic scene object image information to obtain a basic event recognition result, and determining a basic event video indicated by the basic event recognition result from the security videos;

[0123] Basic scene objects, such as delivery personnel, family members, property, children, pets, and other basic scene objects in home life scenes;

[0124] Illustratively, the security video is subjected to basic scene object detection by the basic event identification model: objects in the basic scene, such as people, vehicles, animals, etc., are detected and identified from the security video, and then image feature information of the security video image frames in which the basic scene objects appear is extracted to obtain basic scene object image information, which can be used to locate and mark the objects in the security video image frames; then, the basic event identification is performed based on the basic scene object image information: the identified basic scene object image information is used to classify and identify events to obtain basic event identification results, which are usually specific events such as delivery personnel, family members, property personnel, crying, pet appearance, and user watching TV in the home life scene.

[0125] The basic event identification result is used to indicate which part of the security video is the event video segment of the basic event; according to the basic event identification result, the corresponding event video segment is determined from the security video, such as locating the specific time period of the basic event in the security video, and then determining the event video segment.

[0126] S208: performing video cover screening processing on the basic event video by the basic event identification model to obtain an event video cover, and outputting event video cover information;

[0127] The event video cover information includes an event video cover image and video cover meta information corresponding to the event video cover image. The video cover meta information is information describing the video cover image in the security video, such as a time point corresponding to the video cover image, a video time period of the video cover image corresponding to the basic event, device information of the video cover image, a home location corresponding to the cover, an event type, etc.

[0128] The event video cover is obtained by performing video cover screening processing on the basic event video, which can select the most representative frame from the determined basic event video as the cover of the event video. The trained basic event identification model analyzes the key frame in the event video, selects the event video cover that best represents the event, and generates video cover meta information after determining the event video cover, thereby combining the event video cover image and the video cover meta information to obtain the event video cover information.

[0129] Optionally, for the smart home device, the basic event video in the security video can not be directly extracted, but the event video cover information can be uploaded, and the basic event video can be extracted by the cloud service platform subsequently;

[0130] Optionally, the smart home device can directly extract the basic event video to replace the security video, and upload the basic event video to the cloud service platform as the security video.

[0131] In an embodiment, the event video cover obtained by the video cover screening processing of the basic event video by the basic event recognition model can be:

[0132] The video frame scene information of the basic event video frame contained in the basic event video is extracted by the basic event recognition model, the basic event video frame is scored based on the video frame scene information, and at least one frame of event video cover is determined from the basic event video frame based on the basic scene score.

[0133] S210: The event video cover information and the security video are sent to the cloud service platform, so that the cloud service platform performs event scene analysis based on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on the security video based on the event scene classification information and the event video cover information.

[0134] For details, refer to the method steps of other embodiments of the present specification, which will not be described here.

[0135] In one or more embodiments of the present application, the smart home device collects the security video of the environment, uses a basic event recognition model to recognize the basic event of the security video to obtain event video cover information, and sends the event video cover information and the security video to the cloud service platform. The service platform assists the platform to perform event scene analysis based on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on the security video based on the event scene classification information and the event video cover information. After the video scene is accurately classified based on the event video cover and the event scene classification information, the user can access the cloud service platform, quickly query the interested video for video review according to the video classification, thereby improving the convenience and efficiency of video review of the user, and strengthening the security protection of the family, making the smart home device application more perfect and practical.

[0136] In one embodiment, as Figure 3 shown, a video processing method is particularly proposed, which can be implemented by relying on a computer program and can be run on a video processing device based on the von Neumann architecture. The computer program can be integrated in an application or run as an independent tool application. The video processing device can be a cloud service platform.

[0137] Specifically, the video processing method includes:

[0138] S302: receiving event video cover information uploaded by the smart home device and security video, the event video cover information being information obtained by the smart home device from performing basic event identification on the security video using a basic event identification model;

[0139] According to some embodiments, the smart home device can start the video collection function when it identifies that there is a monitoring object (such as a person, an animal, or a movable object) in the environment, collect security video of the environment, and use a basic event identification model to perform basic event identification on the security video to obtain event video cover information. The smart home device sends the event video cover information and the security video to the cloud service platform.

[0140] The service platform receives the event video cover information and the security video uploaded by the smart home device.

[0141] S304: performing event scene analysis using an event scene analysis large model based on the event video cover information to obtain event scene classification information, and performing video scene classification processing on the security video based on the event scene classification information and the event video cover information.

[0142] The event scene classification information is an event scene classification label obtained by the cloud event scene analysis large model after performing event scene classification on the event scene video (also referred to as basic event video) in the security video. Subsequently, the security video or the event scene video in the security video can be labeled, that is, the event scene classification label is annotated, and the security video or the event scene video in the security video is divided into a scene classification set corresponding to the event scene classification label. In one possible implementation, the event scene analysis large model can be a basic large language model.

[0143] In one possible implementation, the event scene analysis large language model can be obtained by performing model training on a basic large language model for event scene analysis. The basic large language model (LLM) is an artificial intelligence content generation model designed to understand and generate human language. The basic large language model is trained on a large amount of data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and the like.

[0144] Optionally, the basic large language model can be a common AIGC model, such as the Wenxin Yiyang large model, the Tongyi Qianwen large model, the GPT series large model, and the like.

[0145] Optionally, a pre-trained base large language model can be obtained, and the base large language model is adapted to the event scene analysis scene. The initial event scene analysis large language model is created by obtaining the base large language model, and sample event video cover information in the new event scene analysis scene is obtained. The sample event video cover information includes a sample event video cover image and sample video cover meta information corresponding to the sample event video cover image. The initial event scene analysis large language model is fine-tuned and trained using the sample event video cover information. After the model fine-tuning and training is completed, an event scene analysis large language model adapted to the event scene analysis scene is obtained.

[0146] Illustratively, an event scene analysis template can be used to generate an event scene analysis prompt word based on event video cover information. The event scene analysis prompt word is used to instruct the event scene analysis large model to perform event scene analysis based on the event video cover information to obtain event scene classification information. The event scene analysis prompt word and the event video cover information are input into the event scene analysis large model to perform event scene analysis to obtain event scene classification information. The event video cover image in the event video cover information and the video cover meta information corresponding to the event video cover image are determined. The event scene video is extracted from the security video based on the video cover meta information. The event video cover image is used as the video cover of the event scene video. The event scene video is associated with the event scene classification set corresponding to the event scene classification information. The user can access the cloud service platform (e.g., through a terminal) to quickly query the interested video based on the video classification of each event scene classification set for video review, which can facilitate the user to review more specific scene videos.

[0147] In one or more embodiments of the present application, the service platform obtains event video cover information and security video uploaded by the smart home device, and then performs event scene analysis based on the event video cover information using an event scene analysis large model to obtain event scene classification information. The security video is processed based on the event scene classification information and the event video cover information. After the video scene is accurately classified based on the event video cover and the event scene classification information, the user can access the cloud service platform to quickly query the interested video based on the video classification for video review, thereby improving the convenience and efficiency of video review, and enhancing the security protection of the family, making the smart home device application more perfect and practical.

[0148] Please refer to Figure 4 , Figure 4 is a flowchart of another embodiment of a video processing method proposed in the present application. Specifically:

[0149] S402: receive event video cover information uploaded by the smart home device and security video, wherein the event video cover information is information obtained by the smart home device from the security video based on a basic event identification model;

[0150] For details, refer to the method steps of other embodiments of the present specification, which will not be described here.

[0151] S404: obtain event video cover image and video cover meta information corresponding to the event video cover image from the event video cover information;

[0152] For details, refer to the method steps of other embodiments of the present specification, which will not be described here.

[0153] S406: construct event scene analysis prompt words for the event video cover image based on the video cover meta information;

[0154] The event scene analysis prompt words for the event video cover image can be constructed based on the video cover meta information, which is information describing the video cover image in the security video, such as the time point corresponding to the video cover image, the video time period of the video cover image corresponding to the basic event, the collection device information of the video cover image, the home location corresponding to the cover, the event type, and the like. The event scene analysis prompt words are generated in combination with the video cover meta information and the event scene analysis template.

[0155] S408: input the event scene analysis prompt words and the event video cover image into an event scene analysis large model for event scene analysis, and output event scene classification information.

[0156] The event video cover image is combined with the event scene analysis prompt words fused with the video cover meta information to provide rich context information for the large model, so as to enhance the analysis accuracy of the event scene.

[0157] Pre-set key scene classification labels such as old people, babies / children, men, women, doing homework, watching mobile phones, dining, and multiple people, the event scene analysis large model will perform text generation conversion on the event video cover image based on the event scene analysis prompt words to obtain event cover description information, then extract one or more key semantic words (which can be understood as key semantic information) according to the event cover description information, perform event scene classification on the key semantic information to obtain event scene classification information, which can be understood as one or more event scene classification labels divided for the event video cover image, and then the related video corresponding to the event video cover image needs to be labeled with the event scene classification label in the subsequent process.

[0158] In an implementable embodiment, the event scene analysis prompt word and the event video cover image are input into an event scene analysis large model, the event cover description information is obtained by performing image-to-text conversion on the event video cover image based on the event scene analysis prompt word through the event scene analysis large model, the key semantic information is obtained by performing key semantic analysis on the event cover description information, the event scene classification information is obtained by performing event scene classification on the key semantic information, and the event scene classification information is output through the scene analysis large model. The key semantic information extracted from the video cover description can be effectively used for accurate event scene classification, thereby enhancing the intelligence and automation level of event playback. This not only improves the response speed of event playback processing, but also improves the accuracy of event processing.

[0159] The image-to-text conversion of the event video cover image based on the event scene analysis prompt word through the event scene analysis large model is that the event scene analysis large model uses the event scene analysis prompt word and the event video cover image to generate text information describing the event scene corresponding to the event video cover image, and the text information is the event cover description information.

[0160] The key semantic analysis of the event cover description information is that the key semantic elements are extracted from the generated event cover description information to obtain key semantic information, such as named entities, actions or activities, event keywords, actors, objects, and time.

[0161] The event scene classification of the key semantic information is that the key semantic information is converted into semantic vector features, the semantic vector features are matched with feature labels of key scene classification labels, if there is a matched key scene classification label, the event scene classification label corresponding to the matched key scene classification label is obtained, in addition, if there is no matched target key scene classification label, a target key scene label is newly created based on the semantic vector features, and finally the event scene classification information including the target key scene classification labels is output.

[0162] For example, the model training process of the event scene analysis large model is explained as follows:

[0163] A2: An initial event scene analysis large model for event scene analysis is created using a basic large language model.

[0164] Optionally, a basic large language model is obtained, and based on the basic large language model, an initial event scene analysis large language model including at least an event scene analysis scene adaptation module and a large language generation module is created.

[0165] The basic large language model that has been trained can be obtained, the basic large language model is adapted to the event scene analysis scene to obtain an initial event scene analysis large language model, and the event scene analysis large language model is usually directly applied to event scene classification in the event scene analysis scene. It is difficult to adapt to new event scene analysis scenes. Based on this, the initial event scene analysis large language model is created by obtaining the basic large language model, and sample data in the new event scene analysis scene is obtained. Since the basic large language model is usually an open source AIGC model that has been trained and has content generation capability, in this specification, only the adaptation of the event scene analysis scene to the basic large language model is required. Specifically, the initial event scene analysis large language model can be fine-tuned and trained using sample data. After the model fine-tuning training is completed, an event scene analysis large language model adapted to the event scene analysis scene is obtained.

[0166] Model creation: obtain a basic large language model, create an event scene analysis adaptation module for the event scene analysis scene and a large language generation module based on the basic large language model, and compose an initial event scene analysis large language model based on the large language generation module and the event scene analysis scene adaptation module.

[0167] A4: obtaining sample event video cover information, and labeling event scene classification information tags on the sample event video cover information;

[0168] The sample event video cover information is event video cover information in the large model training stage.

[0169] A6: using the sample event video cover information to perform at least one round of model training on the initial event scene analysis large model. In the model training process, the initial event scene analysis large model is used to perform event scene analysis processing on the sample event video cover information to obtain predicted event scene classification information. Based on the predicted event scene classification information and the event scene classification information tags, the model parameters of the initial event scene analysis large model are adjusted to obtain an event scene analysis large model after model training.

[0170] The model training process comprises: inputting sample event video cover information into the initial event scene analysis large language model to perform at least one round of model training to obtain predicted event scene classification information; calculating a comprehensive model loss based on the predicted event scene classification information and event scene classification information labels; performing model parameter adjustment on the event scene analysis scene adaptation module in the initial event scene analysis large language model based on the comprehensive model loss, and controlling the model parameters of the large language generation module to be unchanged. Until a model training end condition is met, a large language generation module and an event scene analysis scene adaptation module are obtained, model fusion of the large language generation module and the event scene analysis scene adaptation module is completed, and a trained event scene analysis large language model is obtained.

[0171] The model fusion of the large language generation module and the event scene analysis scene adaptation module comprises: fusing the model structure layer weights of the event scene analysis scene adaptation module and the large language generation module, determining a target model structure layer corresponding to the model structure layer weights in the large language generation module, performing parameter fusion on the model structure layer parameters and the model structure layer weights of the target model structure layer, and performing parameter updating on the model structure layer based on the model structure layer weights only for part of the target model structure layers in all model structure layers of the basic large language model and having the model structure layer weights. The reference updating process of all model structure layer weights is completed in this way, so that the event scene analysis large language model is obtained.

[0172] Optionally, the model end training condition of the model can comprise, for example, that the value of a loss function is less than or equal to a preset loss function threshold, the number of iterations reaches a preset number threshold, and the like. The specific model end training condition can be determined based on actual conditions, and is not limited here.

[0173] S410: determining an event video cover image in the event video cover information and video cover meta information corresponding to the event video cover image;

[0174] S412: extracting an event scene video from the security video based on the video cover meta information, and taking the event video cover image as a video cover of the event scene video;

[0175] The video cover meta information is information describing the video cover image in the security video, such as a time point corresponding to the video cover image, a video time period of a basic event corresponding to the video cover image, acquisition device information of the video cover image, a home location corresponding to the cover, an event type, and the like. Based on this, the video time period of the basic event in the video cover meta information can be used to extract an event scene video from the security video, and the event video cover image can be taken as a video cover of the event scene video.

[0176] S414: associate the event scene video with the event scene classification set corresponding to the event scene classification information.

[0177] Exemplarily, the event scene analysis prompt word and the event video cover image are input into an event scene analysis large model, and the event scene classification information is output by performing event scene analysis on each event cover by using the event scene analysis large model. Then, intelligent classification is performed based on the event scene classification information, so as to associate the event scene video with the event scene classification set corresponding to the event scene classification information.

[0178] Optionally, after the event scene classification information of the event scene video is determined, the preset event scene classification information is one or more event scene classification labels, such as Figure 5 as shown, Figure 5 is an interface schematic diagram of intelligent cover search. A cloud service platform provides a fast search function for a large number of event scene videos. As shown in Figure 5 , on the intelligent cover search interface, a plurality of preset event scene classification labels are defined, such as a smiling face label, a pet appearance label, a mobile phone watching label, a homework writing label, a television watching label, a gathering and dining label, a car label, and a plurality of person classification labels. Each preset event scene classification label corresponds to an associated sample event scene video.

[0179] Further, after the event scene classification information of the event scene video is determined, the event scene classification information is one or more event scene classification labels. First, it is detected whether all preset event scene classification labels exist in the current event scene classification label. If the event scene classification label exists, the event scene video is directly associated with the event scene classification label. If the event scene classification label does not exist, a current event scene classification label that is not set can be newly established, the event scene video is associated with the newly established event scene classification label, or a candidate event scene video queue of the current event scene classification label is established, the current event scene video is associated with the candidate event scene video queue, and it is monitored whether the queue video number of the candidate event scene video queue is greater than a newly established label threshold. If yes, a preset event scene classification label corresponding to the candidate event scene video queue is newly established.

[0180] Exemplarily, a user can access an intelligent cover search interface of a cloud service platform. The user can input an event video search keyword in the intelligent cover search bar. The cloud service platform matches the preset event scene classification labels with the event video search keyword, obtains a target event scene classification label matched with the event video search keyword, and then displays one or more target event scene videos associated with the target event scene classification label. As shown in Figure 6 , Figure 6is a scene schematic diagram of intelligent cover search, a user inputs "red clothes person" on an intelligent cover search bar on an intelligent cover search interface of a cloud service platform, the cloud service platform matches "red clothes person" with preset event scene classification labels, determines that there is a target event scene classification label matching the "red clothes person", and then displays six videos associated with the target event scene classification label as shown in Figure 6 , realizes quick video review of interest videos according to each event scene classification set after video classification, and facilitates users to review more specific scene videos;

[0181] Optionally, the user can also customize an event scene classification label of interest, which can be used for subsequent targeted viewing of corresponding event videos.

[0182] In one or more embodiments of the present application, the service platform obtains event video cover information and security video uploaded by a smart home device, then performs event scene analysis based on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on the security video based on the event scene classification information and the event video cover information. After accurate video scene classification based on the event video cover and the event scene classification information, the user can access the cloud service platform, quickly query interest videos according to video classification, and review the videos, thereby improving the convenience and efficiency of video review of the user, and also strengthening the security protection of the family, making the smart home device application more perfect and practical.

[0183] The video processing device provided by the embodiments of the present application will be described in detail below. Figure 7 It should be noted that Figure 7 , the video processing device shown in Figures 1-6 is configured to execute the method of the embodiments of the present application. For the purpose of illustration, only parts related to the embodiments of the present application are shown, and specific technical details not disclosed are described with reference to the embodiments shown in Figures 1-6 .

[0184] Please refer to Figure 7 , which shows a structural schematic diagram of the video processing device according to the embodiments of the present application. The video processing device 1 can be realized by software, hardware or a combination of both to become all or part of the device. According to some embodiments, the video processing device 1 includes a video acquisition module 11, an event identification module 12 and a scene analysis module 13, which are specifically configured to:

[0185] The video acquisition module 11 is configured to acquire security video of the environment;

[0186] The event identification module 12 is configured to perform basic event identification on the security video by using a basic event identification model to obtain event video cover information;

[0187] The scene analysis module 13 is configured to send the event video cover information and the security video to a cloud service platform, so that the cloud service platform performs event scene analysis based on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on the security video based on the event scene classification information and the event video cover information.

[0188] In an implementation, the event recognition module 12 is configured to:

[0189] input the security video into a basic event recognition model;

[0190] perform basic scene object detection on the security video by using the basic event recognition model to obtain basic scene object image information, perform basic event recognition based on the basic scene object image information to obtain a basic event recognition result, and determine a basic event video indicated by the basic event recognition result from the security video;

[0191] perform video cover screening processing on the basic event video by using the basic event recognition model to obtain an event video cover, and output event video cover information;

[0192] The event video cover information includes an event video cover image and video cover meta information corresponding to the event video cover image.

[0193] In an implementation, the event recognition module 12 is configured to:

[0194] extract video frame scene information of a basic event video frame included in the basic event video by using the basic event recognition model, perform basic scene scoring on the basic event video frame based on the video frame scene information, and determine at least one frame of event video cover from the basic event video frame based on the basic scene score.

[0195] It should be noted that the video processing apparatus provided in the above embodiments is only used as an example to divide the above functions into different functional modules in the implementation of the video processing method. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the video processing apparatus and the video processing method provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments. Therefore, it is not repeated here.

[0196] The sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0197] Please refer toFigure 8 FIG. 2 shows a structural schematic diagram of a video processing apparatus according to an embodiment of the present application. The video processing apparatus 2 can be realized by software, hardware or a combination of both to become all or part of the apparatus. According to some embodiments, the video processing apparatus 2 comprises a data receiving module 21, a scene analysis module 22, and is specifically used for:

[0198] The data receiving module 21 is configured to receive event video cover information uploaded by a smart home device and security video, wherein the event video cover information is information obtained by using a basic event identification model to perform basic event identification on the security video by the smart home device.

[0199] The scene analysis module 22 is configured to perform event scene analysis on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and perform video scene classification processing on the security video based on the event scene classification information and the event video cover information.

[0200] In a possible implementation, the scene analysis module 22 is configured to:

[0201] obtain an event video cover image and video cover meta information corresponding to the event video cover image from the event video cover information;

[0202] construct an event scene analysis prompt word for the event video cover image based on the video cover meta information;

[0203] input the event scene analysis prompt word and the event video cover image into an event scene analysis large model to perform event scene analysis, and output event scene classification information.

[0204] In a possible implementation, the scene analysis module 22 is configured to:

[0205] add the video cover meta information to an event scene analysis template to obtain an event scene analysis prompt word for the event video cover image.

[0206] In a possible implementation, the scene analysis module 22 is configured to:

[0207] input the event scene analysis prompt word and the event video cover image into an event scene analysis large model;

[0208] perform image-to-text conversion on the event video cover image based on the event scene analysis prompt word by the event scene analysis large model to obtain event cover description information, perform key semantic analysis on the event cover description information to obtain key semantic information, and perform event scene classification on the key semantic information to obtain event scene classification information;

[0209] output the event scene classification information by the scene analysis large model.

[0210] In an implementation, the apparatus 2 is further configured to:

[0211] create an initial event scene analysis large model for event scene analysis by using a basic large language model;

[0212] obtain sample event video cover information, and label event scene classification information tags for the sample event video cover information;

[0213] perform at least one round of model training on the initial event scene analysis large model by using the sample event video cover information, in the model training process, perform event scene analysis processing on the sample event video cover information by using the initial event scene analysis large model to obtain predicted event scene classification information, adjust model parameters of the initial event scene analysis large model based on the predicted event scene classification information and the event scene classification information tags, and obtain the event scene analysis large model after model training.

[0214] In an implementation, the scene analysis module 22 is configured to:

[0215] determine event video cover images in the event video cover information and video cover meta information corresponding to the event video cover images;

[0216] extract event scene videos from the security videos based on the video cover meta information, and use the event video cover images as video covers of the event scene videos;

[0217] associate the event scene videos with event scene classification sets corresponding to the event scene classification information.

[0218] The embodiments of the present application further provide a computer storage medium, which can store a plurality of instructions, and the instructions are suitable for being loaded and executed by a processor to implement the video processing method of the embodiments of the video processing method as shown in the above Figures 1-6 The specific implementation process can refer to the specific description of the embodiments of the video processing method as shown in the above Figures 1-6 The specific implementation process can refer to the specific description of the embodiments of the video processing method as shown in the above

[0219] The embodiments of the present application further provide a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the video processing method of the embodiments of the video processing method as shown in the above Figures 1-6 The specific implementation process can refer to the specific description of the embodiments of the video processing method as shown in the above Figures 1-6 The specific implementation process can refer to the specific description of the embodiments of the video processing method as shown in the above

[0220] Please refer toFigure 9 Fig. 1 shows a structural block diagram of an electronic device according to an example embodiment of the present application. The electronic device in the present application can include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 can be connected through the bus 150.

[0221] The processor 110 can include one or more processing cores. The processor 110 connects various parts within the entire electronic device by using various interfaces and lines, performs various functions of the electronic device 100 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and calling data stored in the memory 120. Alternatively, the processor 110 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA). The processor 110 can integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes an operating system, a user interface, and an application program; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 110, but be implemented separately through a communication chip.

[0222] The memory 120 can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 120 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing each of the methods described below, etc., and the operating system can be an Android system, an IOS system developed by Apple Inc., a system developed based on the Android system or the IOS system, or other systems. The data storage area can also store data created by the electronic device during use, such as a phone book, audio and video data, chat record data, etc.

[0223] Referring to Figure 10 As shown, the memory 120 can be divided into an operating system space and a user space, and the operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve good running effects, the operating system allocates corresponding system resources to different third-party applications. However, there are also differences in the demand for system resources in different application scenarios in the same third-party application, for example, in the local resource loading scenario, the third-party application has a higher requirement for the disk reading speed, and in the animation rendering scenario, the third-party application has a higher requirement for the GPU performance. However, the operating system and the third-party application are independent of each other, and the operating system often cannot timely perceive the current application scenario of the third-party application, resulting in that the operating system cannot perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0224] In order to enable the operating system to distinguish the specific application scenario of the third-party application, it is necessary to open up the data communication between the third-party application and the operating system, so that the operating system can obtain the current scenario information of the third-party application at any time, and then perform targeted system resource adaptation based on the current scenario.

[0225] Taking the operating system as an Android system for example, the programs and data stored in the memory 120 are as follows Figure 11As shown, the memory 120 can store the Linux kernel layer 320, the system runtime library layer 340, the application framework layer 360, and the application layer 380. The Linux kernel layer 320, system runtime library layer 340, and application framework layer 360 belong to the operating system space, while the application layer 380 belongs to the user space. The Linux kernel layer 320 provides low-level drivers for various hardware components of the electronic device, such as display drivers, audio drivers, camera drivers, Bluetooth drivers, Wi-Fi drivers, and power management. The system runtime library layer 340 provides support for key features of the Android system through several C / C++ libraries. For example, the SQLite library provides database support, the OpenGL / ES library provides 3D graphics support, and the Webkit library provides browser kernel support. The system runtime library layer 340 also provides the Android runtime library, which mainly provides core libraries that allow developers to write Android applications using the Java language. The Application Framework Layer 360 provides various APIs that may be used when building applications. Developers can also use these APIs to build their own applications, such as activity management, window management, view management, notification management, content provider, package management, call management, resource management, and location management. At least one application runs in the Application Layer 380. These applications can be native applications that come with the operating system, such as contacts, SMS, clock, and camera apps; or third-party applications developed by third-party developers, such as games, instant messaging, and photo editing apps.

[0226] Taking the operating system as an example (iOS), the programs and data stored in memory 120 are as follows: Figure 12As shown, the IOS system includes: a core operating system layer 420, a core service layer 440, a media layer 460, and a Cocoa Touch layer 480. The core operating system layer 420 includes an operating system kernel, drivers, and low-level hardware abstractions. The core service layer 440 provides more abstracted program frameworks for the application programs to use. The media layer 460 provides interfaces for audio and video related functions. The Cocoa Touch layer 480 provides various interface related frameworks for application development. The Cocoa Touch layer 480 is responsible for touch interaction on the electronic device. For example, the local notification service, the remote push service, the advertisement framework, the game tool framework, the message user interface (UI) framework, the user interface UIKit framework, and the map framework.

[0227] In Figure 12 In the framework shown, the frameworks related to most of the application programs include, but are not limited to, the Foundation framework in the core service layer 440 and the UIKit framework in the Cocoa Touch layer 480. The Foundation framework provides many basic object classes and data types, and provides the most basic system services for all application programs, and is independent of the UI. The UIKit framework provides basic UI class libraries for creating touch-based user interfaces. The iOS application program can provide UI based on the UIKit framework, so it provides the basic framework of the application program for building user interfaces, drawing, processing and user interaction events, responding to gestures, and the like.

[0228] The way and principle of implementing the third-party application program to communicate with the operating system data in the IOS system can refer to the Android system, which will not be described herein.

[0229] The input device 130 is configured to receive input instructions or data, and the input device 130 includes but is not limited to a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is configured to output instructions or data, and the output device 140 includes but is not limited to a display device and a speaker. In an example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are a touch display screen configured to receive a touch operation of a user using a finger, a touch pen, or any suitable object on or near the touch display screen, and display a user interface of each application. The touch display screen is usually arranged on a front panel of the electronic device. The touch display screen can be designed as a full screen, a curved screen, or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, and the present application does not limit the combination.

[0230] In addition, those skilled in the art can understand that the structure of the electronic device shown in the above-described drawings does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than those shown in the drawings, or combine certain components, or different component arrangements. For example, the electronic device further includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module, and the like, which are not described herein.

[0231] In the embodiments of the present application, the execution subject of each step can be the electronic device introduced above. Alternatively, the execution subject of each step is an operating system of the electronic device. The operating system can be an Android system, an IOS system, or other operating systems, and the present application does not limit the operating system.

[0232] The electronic device of the embodiments of the present application can further have a display device installed thereon. The display device can be various devices capable of realizing display functions, such as a cathode ray tube display (CR), a light-emitting diode display (LED), an electronic ink screen, a liquid crystal display (LCD), a plasma display panel (PDP), and the like. A user can use the display device on the electronic device 101 to view displayed text, images, video, and the like. The electronic device can be a smart phone, a tablet computer, a game device, an AR (Augmented Reality) device, a car, a data storage device, an audio playback device, a video playback device, a notebook computer, a desktop computing device, a wearable device such as an electronic watch, electronic glasses, an electronic helmet, an electronic bracelet, an electronic necklace, an electronic clothing, and the like.

[0233] In Figure 9 In the electronic device shown, the processor 110 can be configured to invoke an application stored in the memory 120 and specifically execute the video processing method of one or more embodiments of the present application.

[0234] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium and can include the processes of the above-mentioned embodiments when executed.

[0235] The above disclosure is only the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application. Therefore, equivalent changes made in accordance with the claims of the present application are still within the scope of the present application.

Claims

1. A method of video processing, the method comprising: The method is applied to a smart home device and comprises the following steps: Collecting a security video of an environment; Inputting the security video into a basic event recognition model, performing basic event recognition on the security video by using the basic event recognition model to obtain event video cover information, and the basic event being a basic event for a basic scene object in a home life scene; Sending the event video cover information and the security video to a cloud service platform, so that the cloud service platform performs event scene analysis on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and performs video scene classification processing on a basic event video in the security video based on the event scene classification information and the event video cover information, and the event scene classification information being an event scene classification label obtained by the event scene analysis large model after performing event scene classification on the basic event video in the security video; The basic event recognition model is used to perform basic event recognition on the security video to obtain event video cover information, which comprises the following steps: Performing basic scene object detection on the security video by using the basic event recognition model to obtain basic scene object image information, performing basic event recognition based on the basic scene object image information to obtain a basic event recognition result, and determining a basic event video indicated by the basic event recognition result from the security video; Performing video cover screening processing on the basic event video by using the basic event recognition model to obtain an event video cover, and outputting event video cover information; The event video cover information comprises an event video cover image and video cover meta information corresponding to the event video cover image.

2. The method of claim 1, wherein, The basic event recognition model is used to perform video cover screening processing on the basic event video to obtain an event video cover, which comprises the following steps: Extracting video frame scene information of a basic event video frame contained in the basic event video by using the basic event recognition model, performing basic scene scoring on the basic event video frame based on the video frame scene information, and determining at least one frame of event video cover from the basic event video frame based on the basic scene scoring.

3. A method for video processing, the method comprising: The method is applied to a cloud service platform and comprises the following steps: Receiving event video cover information and a security video uploaded by a smart home device, the event video cover information being an event video cover image obtained after a basic event recognition model performs video cover screening processing on a basic event video and video cover meta information corresponding to the event video cover image, the basic event video being a video indicated by a basic event recognition result, the basic event recognition result being a result obtained by the basic event recognition model based on basic scene object image information, and the basic scene object image information being information obtained by the basic event recognition model after performing basic scene object detection on the security video; The event scene classification information is obtained by using an event scene analysis large model based on the event video cover information, and the basic event video in the security video is subjected to video scene classification processing based on the event scene classification information and the event video cover information.

4. The method of claim 3, wherein, The event scene classification information is obtained by using an event scene analysis large model based on the event video cover information, and the basic event video in the security video is subjected to video scene classification processing based on the event scene classification information and the event video cover information. The event scene classification information is obtained by using an event scene analysis large model based on the event video cover information, and the basic event video in the security video is subjected to video scene classification processing based on the event scene classification information and the event video cover information. The event scene classification information is obtained by using an event scene analysis large model based on the event video cover information, and the basic event video in the security video is subjected to video scene classification processing based on the event scene classification information and the event video cover information. The event scene classification information is obtained by using an event scene analysis large model based on the event video cover information, and the basic event video in the security video is subjected to video scene classification processing based on the event scene classification information and the event video cover information.

5. The method of claim 4, wherein, The event scene classification information is obtained by using an event scene analysis large model based on the event video cover information, and the basic event video in the security video is subjected to video scene classification processing based on the event scene classification information and the event video cover information. The event scene classification information is obtained by using an event scene analysis large model based on the event video cover information, and the basic event video in the security video is subjected to video scene classification processing based on the event scene classification information and the event video cover information.

6. The method of claim 4, wherein, The method further comprises: creating an initial event scene analysis large model for event scene analysis by using a basic large language model; obtaining sample event video cover information, and labeling event scene classification information tags for the sample event video cover information; performing at least one round of model training on the initial event scene analysis large model by using the sample event video cover information, in the model training process, performing event scene analysis processing on the sample event video cover information by using the initial event scene analysis large model to obtain predicted event scene classification information, adjusting model parameters of the initial event scene analysis large model based on the predicted event scene classification information and the event scene classification information tags, and obtaining the event scene analysis large model after model training.

7. The method of claim 3, wherein, The event scene classification information is obtained by using an event scene analysis large model based on the event video cover information, and the basic event video in the security video is subjected to video scene classification processing based on the event scene classification information and the event video cover information. The event scene classification information is obtained by using an event scene analysis large model based on the event video cover information, and the basic event video in the security video is subjected to video scene classification processing based on the event scene classification information and the event video cover information. ​ ​ 8. The method of claim 3, wherein, ​ ​ extracting a basic event video from the security video based on the video cover meta information, and taking the event video cover image as a video cover of the basic event video; associating the basic event video with an event scene classification set corresponding to the event scene classification information.

9. A video processing apparatus, comprising: The device is applied to a smart home device and includes: a video collection module configured to collect a security video of an environment; an event identification module configured to input the security video into a basic event identification model, perform basic event identification on the security video by using the basic event identification model to obtain event video cover information, and identify a basic event in a home life scene for a basic scene object; a scene analysis module configured to send the event video cover information and the security video to a cloud service platform, so that the cloud service platform performs event scene analysis by using an event scene analysis large model based on the event video cover information, performs video scene classification processing on a basic event video in the security video based on the event scene classification information and the event video cover information, and obtains event scene classification information by classifying the basic event video in the security video based on the event scene analysis large model; the event identification module is configured to: perform basic scene object detection on the security video by using the basic event identification model to obtain basic scene object image information, perform basic event identification based on the basic scene object image information to obtain a basic event identification result, and determine a basic event video indicated by the basic event identification result from the security video; perform video cover screening processing on the basic event video by using the basic event identification model to obtain an event video cover, and output event video cover information; wherein the event video cover information includes an event video cover image and video cover meta information corresponding to the event video cover image.

10. The apparatus of claim 9, wherein, the event identification module is configured to: extract video frame scene information of a basic event video frame included in the basic event video by using the basic event identification model, perform basic scene scoring on the basic event video frame based on the video frame scene information, and determine at least one frame of event video cover from the basic event video frame based on the basic scene scoring.

11. A video processing apparatus, comprising: The device is applied to a cloud service platform and includes: a data receiving module configured to receive event video cover information and a security video uploaded by a smart home device, the event video cover information being obtained by inputting the security video into a basic event identification model by the smart home device, performing basic event identification on the security video by using the basic event identification model to obtain a basic event video, performing video cover screening processing on the basic event video to obtain an event video cover image, and obtaining video cover meta information corresponding to the event video cover image, and the basic event being a basic event in a home life scene for a basic scene object. A scene analysis module is configured to perform event scene analysis on the event video cover information by using an event scene analysis large model to obtain event scene classification information, and perform video scene classification processing on the basic event video in the security video based on the event scene classification information and the event video cover information. The event scene classification information is an event scene classification label obtained by the event scene analysis large model after performing event scene classification on the basic event video in the security video.

12. The apparatus of claim 11, wherein, The scene analysis module is configured to: obtain an event video cover image and video cover meta information corresponding to the event video cover image from the event video cover information; construct an event scene analysis prompt word for the event video cover image based on the video cover meta information; input the event scene analysis prompt word and the event video cover image into an event scene analysis large model to perform event scene analysis, and output event scene classification information.

13. The apparatus of claim 12, wherein, The scene analysis module is configured to: add the video cover meta information to an event scene analysis template to obtain an event scene analysis prompt word for the event video cover image.

14. The apparatus of claim 12, wherein, The scene analysis module is configured to: input the event scene analysis prompt word and the event video cover image into an event scene analysis large model; perform image-to-text conversion on the event video cover image based on the event scene analysis prompt word by using the event scene analysis large model to obtain event cover description information, perform key semantic analysis on the event cover description information to obtain key semantic information, and perform event scene classification on the key semantic information to obtain event scene classification information; output the event scene classification information by using the scene analysis large model.

15. The apparatus of claim 11, wherein, The device is further configured to: create an initial event scene analysis large model for event scene analysis by using a basic large language model; obtain sample event video cover information, and label event scene classification information tags for the sample event video cover information; perform at least one round of model training on the initial event scene analysis large model by using the sample event video cover information. During the model training process, perform event scene analysis on the sample event video cover information by using the initial event scene analysis large model to obtain predicted event scene classification information, adjust model parameters of the initial event scene analysis large model based on the predicted event scene classification information and the event scene classification information tags, and obtain an event scene analysis large model after model training.

16. The apparatus of claim 11, wherein, The scene analysis module is configured to: determine an event video cover image in the event video cover information and video cover meta information corresponding to the event video cover image; extract a basic event video from the security video based on the video cover meta information, and use the event video cover image as a video cover of the basic event video; associate the basic event video with an event scene classification set corresponding to the event scene classification information.

17. A computer storage medium, comprising, The computer storage medium stores a plurality of instructions adapted to be loaded and executed by a processor, and the instructions perform the method steps of any one of claims 1-2 or 3-8.

18. An electronic device, comprising: includes: a processor and a memory; wherein the memory stores a computer program adapted to be loaded and executed by the processor and to perform the method steps of any one of claims 1-2 or 3-8.

Images