Method, device, electronic device and medium for detecting home scenes

Abstract

The present application discloses a method, device, electronic device and medium for detecting home scenes. In the present application, a camera device arranged in a smart home terminal can be used to collect home scene video data for a target area; the target video transform domain coefficients of the home scene video data are collected, and the target video transform domain coefficients are input into a preset scene detection network model to detect whether there is a user operation event in the target area; if so, the smart home terminal is controlled to perform an action corresponding to the user operation event. By applying the technical solution of the present application, after the video data is collected by the camera of the smart home, the unreadable video transform domain coefficients of the video data are collected and the original video data is cleared, so that the home scene can be detected later according to the video transform domain coefficient data and the learning model. This avoids the problem of potential safety hazards caused by the readability of the original video data after leakage.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to a method, device, electronic device and medium for detecting home scenes. Background Art

[0002] In smart city application scenarios such as smart homes, visual signals are the main and indispensable source of information in various scenarios, including home monitoring, posture recognition, face recognition, home medical care, autonomous driving, and entertainment live broadcast.

[0003] Among them, although visual signals play an indispensable and important role in smart homes, in various application scenarios, smart home systems often directly use the original visual signal data for intelligent analysis at different processing stages of visual signal data. This also leads to the potential for great privacy and security risks to users if the original video image signal contains sensitive privacy data of users. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device and medium for detecting home scenes, so as to solve the problem in the related art that user privacy content in home scene video data collected by smart home terminals is easily leaked.

[0005] According to one aspect of an embodiment of the present application, a method for detecting a home scene is provided, comprising:

[0006] Using a camera device installed in a smart home terminal to collect home scene video data for a target area;

[0007] Collecting target video transform domain coefficients of the home scene video data, and inputting the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area;

[0008] If so, control the smart home terminal to execute the action corresponding to the user operation event.

[0009] Optionally, in another embodiment of the method of the present application, the target video transform domain coefficients for collecting the home scene video data include:

[0010] Collecting transform domain coefficients of coding units CU of home scene video data; and, collecting transform domain coefficients of the largest coding unit LCU of home scene video data; and, collecting transform domain coefficients of single frames of home scene video data;

[0011] The CU transform domain coefficients, the LCU transform domain coefficients and the single frame transform domain coefficients are combined as the target video transform domain coefficients.

[0012] Optionally, in another embodiment of the method of the present application, the target video transform domain coefficients for collecting the home scene video data include:

[0013] Decoding the home scene video data to obtain decoded video data;

[0014] The target video transform domain coefficients of the decoded video data are acquired.

[0015] Optionally, in another embodiment of the above method of the present application, after acquiring the target video transform domain coefficients of the home scene video data, the method further includes:

[0016] The home scene video data is cleared.

[0017] Optionally, in another embodiment of the method of the present application, inputting the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area includes:

[0018] According to the identification of the target video transform domain coefficients, determining whether there are human features in the target area where the smart home terminal is currently located, the human features including at least one of size features, color features and contour features;

[0019] If it is determined that the human body feature exists, the target video transform domain coefficient is input into a preset scene detection network model to detect the user operation event.

[0020] Optionally, in another embodiment of the method of the present application, after controlling the smart home terminal to perform an action corresponding to the user operation event, the method further includes:

[0021] Recording the association relationship and associated data between the target video transform domain coefficient and the action of executing the user operation event;

[0022] After the association relationship and the association data are combined into model training data, the model training data is uploaded to the cloud, and the model training data is used to train the scene detection network model.

[0023] Optionally, in another embodiment of the method of the present application, after merging the association relationship and the association data into model training data and uploading the model training data to the cloud, the method further includes:

[0024] After receiving the model update instruction, receiving the updated scene detection network model transmitted from the cloud; and clearing the scene detection network model originally stored in the smart home terminal.

[0025] According to another aspect of the embodiments of the present application, a device for detecting a home scene is provided, characterized in that it includes:

[0026] The first acquisition module is configured to use a camera device provided in a smart home terminal to acquire home scene video data for a target area;

[0027] A second acquisition module is configured to acquire target video transform domain coefficients of the home scene video data, and input the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area;

[0028] The execution module is configured to control the smart home terminal to execute the action corresponding to the user operation event, if it exists.

[0029] According to another aspect of the embodiments of the present application, an electronic device is provided, including:

[0030] A memory for storing executable instructions; and

[0031] The display is used to execute the executable instructions together with the memory to complete the operation of any of the above-mentioned methods for detecting home scenes.

[0032] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided for storing computer-readable instructions, which, when executed, perform the operations of any of the above-mentioned methods for detecting home scenes.

[0033] In the present application, a camera device installed in a smart home terminal can be used to collect home scene video data for a target area; collect target video transform domain coefficients of the home scene video data, and input the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area; if so, control the smart home terminal to perform an action corresponding to the user operation event. By applying the technical solution of the present application, after the video data is collected by the camera of the smart home, the unreadable video transform domain coefficients of the video data are collected and the original video data is cleared, so that the home scene can be detected later based on the video transform domain coefficient data and the learning model. This avoids the problem of potential safety hazards caused by the readability of the original video data after leakage.

[0034] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0036] The present application can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0037] Figure 1 A schematic diagram of a method for detecting home scenes proposed in this application;

[0038] Figure 2 A schematic diagram of the process of converting home scene video data into video transform domain coefficients proposed in this application;

[0039] Figure 3 A schematic diagram of a process for detecting user operation events using a scene detection network model proposed in this application;

[0040] Figure 4 A schematic diagram of a model operation flow for detecting user operation events using a scene detection network model proposed in this application;

[0041] Figure 5 This is a schematic diagram of the structure of an electronic device for detecting home scenes proposed in this application;

[0042] Figure 6 This is a schematic diagram of the structure of an electronic device for detecting home scenes proposed in this application. DETAILED DESCRIPTION

[0043] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0044] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0045] The following description of at least one exemplary embodiment is merely illustrative in nature and is not intended to limit the present application, its application, or uses.

[0046] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0047] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0048] In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0049] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0050] Combine the following Figure 1-Figure 4 To describe the method for detecting home scenes according to an exemplary embodiment of the present application. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0051] The present application also proposes a method, device, electronic device and medium for detecting home scenes.

[0052] Figure 1 The flowchart of a method for detecting a home scene according to an embodiment of the present application is schematically shown. Figure 1 As shown, the method includes:

[0053] S101, using a camera device installed in a smart home terminal to collect home scene video data for a target area.

[0054] S102, collecting target video transform domain coefficients of home scene video data, and inputting the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area.

[0055] S103: If yes, control the smart home terminal to execute the action corresponding to the user operation event.

[0056] In related technologies, in smart city application scenarios such as smart homes, visual signals are the main and indispensable source of information in various scenarios, including home monitoring, gesture recognition, face recognition, home medical care, autonomous driving, and entertainment live broadcasting. With visual signals as the main body, the current mainstream smart home framework has been established around users, data, software and hardware systems, and necessary AI and IoT underlying technologies, which are becoming more and more convenient for people's daily lives and an important part of intelligent and smart life.

[0057] Although visual signals play an indispensable and important role in smart homes, in various application scenarios, smart home systems often directly use raw visual signal data for intelligent analysis at different stages of visual signal data processing. Obviously, once the user's sensitive privacy data in the raw visual signal is leaked at a certain stage, it will inevitably lead to huge privacy and security risks.

[0058] Among visual signals, video data is particularly important. The way smart home systems process video data inevitably leads to privacy and security risks at different stages. Among them, privacy and security issues in scenarios such as smart homes have received great attention and will become an important challenge for smart homes to be more advanced.

[0059] In view of the typical problems mentioned above, in scenarios such as smart homes, the video data stream desensitization method currently mainly adopts the traditional video encoding processing method to further process the decoded data. It can protect private data to a certain extent and reduce the risk of private data being exposed. However, once private data is leaked at a certain stage before decoding, the private data is directly exposed, which will inevitably lead to great privacy and security risks. This post-decoding processing method does not protect the privacy content to a high degree.

[0060] Based on the above problems, the present application can collect the unreadable video transform domain coefficients of the video data and clear the original video data after collecting the video data through the camera of the smart home, so that the home scene can be detected according to the video transform domain coefficient data and the learning model in the future. This avoids the problem of potential safety hazards caused by the original video data being readable and leaking.

[0061] Furthermore, in order to eliminate the potential security risks of video data being cracked and exposed after being leaked, the present application proposes a processing method of directly clearing the original sensitive video data in scenarios with a large amount of sensitive privacy data such as smart homes, and instead directly extracting the video transform domain coefficients of the video data containing home scenes collected by smart home cameras, so that user instructions for home scenes can be detected subsequently based on the video transform domain coefficients.

[0062] Specifically, Figure 2As shown, the present application processes video data in the form of characteristic signals, which may include converting home scene video data into coding unit CU transform domain coefficients for collecting home scene video data; and collecting the maximum coding unit LCU transform domain coefficients for home scene video data; and collecting single frame transform domain coefficients for home scene video data. Furthermore, for many scenes in smart homes, commonly used functions include recognition, retrieval, tracking, etc. These similar functions do not necessarily need to rely on the original data signal to be processed. And because the video transform domain coefficients usually do not have the characteristics of direct human readability and reversibility, that is to say, even if these video transform domain coefficients are leaked, the person who obtains them cannot translate and understand the meaning of these data, thereby greatly reducing the risk of cracking exposure.

[0063] It should also be noted that, after collecting the target video transform domain coefficients of the home scene video data, the present application also needs to clear the home scene video data, thereby further avoiding the problem of potential safety hazards caused by the leakage of the original video image data.

[0064] In addition, the present application can also perform correlation analysis on unreadable data such as CDVS / CDVA and scene detection network models, and convert them into analysis results for various application scenarios, thus overcoming the difficulties of relying on original data and avoiding the risks of privacy leakage and exposure of original video data.

[0065] Specifically, the present application can also input the video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area. The data generated in the process is still not human-readable and reversible, and is unique. Therefore, this technical means can also prevent data from being cracked and exposed, effectively reducing the risk of privacy content.

[0066] In one way, in order to improve the decision-making of smart homes, e.g. Figure 3 As shown, in the embodiment of the present application, when the smart home detects a user operation event caused by human operation, the visual feature signal and the result of the human operation can be recorded and stored as training data for the scene detection network model. The current data replaces the old data as the latest data to ensure that the total amount of data storage remains unchanged. By using this technology, the processed data is transmitted to the cloud for subsequent device updates and function upgrades. The transmitted data does not depend on the original video data, which can effectively avoid the privacy and security risks caused by the data being intercepted.

[0067] In the present application, a camera device installed in a smart home terminal can be used to collect home scene video data for a target area; collect target video transform domain coefficients of the home scene video data, and input the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area; if so, control the smart home terminal to perform an action corresponding to the user operation event. By applying the technical solution of the present application, after the video data is collected by the camera of the smart home, the unreadable video transform domain coefficients of the video data are collected and the original video data is cleared, so that the home scene can be detected later based on the video transform domain coefficient data and the learning model. This avoids the problem of potential safety hazards caused by the readability of the original video data after leakage.

[0068] Optionally, in another embodiment of the method of the present application, the target video transform domain coefficients for collecting the home scene video data include:

[0069] Collecting transform domain coefficients of coding units CU of home scene video data; and, collecting transform domain coefficients of the largest coding unit LCU of home scene video data; and, collecting transform domain coefficients of single frames of home scene video data;

[0070] The CU transform domain coefficients, the LCU transform domain coefficients and the single frame transform domain coefficients are combined as the target video transform domain coefficients.

[0071] Optionally, in another embodiment of the method of the present application, the target video transform domain coefficients for collecting the home scene video data include:

[0072] Decoding the home scene video data to obtain decoded video data;

[0073] The target video transform domain coefficients of the decoded video data are acquired.

[0074] According to the objectives of this application and the problems to be solved, this application proposes to directly abandon the traditional video decoding processing method in scenarios with more sensitive privacy data such as smart homes, and instead adopt the method of collecting transform domain coefficients during decoding for processing. The specific process may include that in the process of processing video data, the first half is processed by the traditional video decoding method, and the second half collects the various video transform domain coefficients of the video data according to the process (for example, including CU transform domain coefficients, LCU transform domain coefficients and single frame transform domain coefficients). It can be understood that through such a processing process, it is not necessary to fully decode the video stream, that is, no reconstructed file is generated, avoiding the risk of privacy leakage of the reconstructed file data.

[0075] Optionally, in another embodiment of the above method of the present application, after acquiring the target video transform domain coefficients of the home scene video data, the method further includes:

[0076] The home scene video data is cleared.

[0077] Optionally, in another embodiment of the method of the present application, inputting the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area includes:

[0078] According to the identification of the target video transform domain coefficients, determining whether there are human features in the target area where the smart home terminal is currently located, the human features including at least one of size features, color features and contour features;

[0079] If it is determined that the human body feature exists, the target video transform domain coefficient is input into a preset scene detection network model to detect the user operation event.

[0080] In this application, the process of inputting the target video transform domain coefficients into a preset scene detection network model for detecting whether there is a user operation event in the target area can include the following steps:

[0081] Among them, the input of the scene detection network model is the target video transform domain coefficient after downsampling, and the downsampling technology is used to eliminate the influence of resolution in home scene images to fix the input size of the neural network model.

[0082] The transform domain coefficients of the target video after downsampling technology are input into the network structure for calculation, wherein the present application does not specifically limit the network structure of the scene detection network model.

[0083] Furthermore, the prediction result output by the scene detection network model may be a prediction result for specifying the category and position of an object, and by identifying the category and position of the object, it is determined whether there is a user operation event in the target area.

[0084] In one approach, the model output can be defined as:

[0085] Y=[P c , b x , b y , b h , b w , C1, C2, C3] T .

[0086] Among them, P c is the probability that there is an object in the graph; b x , by , b h , b w are the position parameters of the corresponding object; C1, C2, C 3为 The category corresponding to the specific object.

[0087] Optionally, in another embodiment of the method of the present application, after controlling the smart home terminal to perform an action corresponding to the user operation event, the method further includes:

[0088] Recording the association relationship and associated data between the target video transform domain coefficient and the action of executing the user operation event;

[0089] After the association relationship and the association data are combined into model training data, the model training data is uploaded to the cloud, and the model training data is used to train the scene detection network model.

[0090] Optionally, in another embodiment of the method of the present application, after merging the association relationship and the association data into model training data and uploading the model training data to the cloud, the method further includes:

[0091] After receiving the model update instruction, receiving the updated scene detection network model transmitted from the cloud; and clearing the scene detection network model originally stored in the smart home terminal.

[0092] In one way, after recording the correlation relationship and correlation data between the target video transform domain coefficients and the actions of executing user operation events, the present application can also use them as model training data to continuously update and train the scene detection network model. It can be understood that in the related art, the transform domain coefficients cannot be used to detect application scenarios such as user operation events through manual operation. Therefore, in order to apply the target transform domain coefficients proposed in this application to specific detection scenarios of smart homes, this application needs to associate the equipment training data in smart furniture with the target transform domain data to generate a network model that can realize target detection based on transform domain data.

[0093] This solution can realize target detection using video decoding transform domain coefficients, avoiding the dependence of smart home products on raw data. The data generated by this technology from the data receiving stage to the data processing stage until the analysis results are generated is not human-readable and reversible, and is unique. Even if the information is leaked, it will not be cracked, greatly reducing the risk of privacy exposure.

[0094] Furthermore, after obtaining a trained scene detection network model that can be used for detection based on transform domain coefficients, in order to avoid the drawback of occupying a large amount of smart home memory due to the excessively large data structure of the scene detection network model, the present application can also compress the model to obtain a corresponding compressed scene detection network model with a smaller data structure.

[0095] Optionally, the present application may compress the scene detection network model by directly compressing it, for example, including the sparsification of the model kernel and the pruning of the model. The sparsification of the kernel requires the support of some sparse computing libraries, and its acceleration effect may be restricted by many factors such as bandwidth and sparsity. In addition, the pruning method of the model requires directly removing unimportant filter parameters from the original model. Since the neural network has a strong adaptive ability and the models with larger data architectures often have more redundancy, after removing some parameters, the performance reduced by the removal of parameters can be restored by retraining. Therefore, it is only necessary to select a suitable pruning method and retraining method to effectively compress the existing model to a large extent, which is the most commonly used method at present.

[0096] Furthermore, after obtaining a compressed scene detection network model with a smaller data structure, the present application can deploy it on a smart home. Thus, the smart home can subsequently use the compressed image detection classification model to identify the home scene video data collected by the camera device, so as to subsequently convert it into the corresponding video transform domain coefficients.

[0097] In addition, the present application can also deploy a scene detection network model with a larger data structure to the server, so as to determine the recognition mode based on the operation status of the smart home, and then specifically select the corresponding detection network model to detect whether there is a user operation event in the target area.

[0098] In one embodiment, the method of realizing smart home scene detection using target transform domain coefficients mentioned in this application can achieve the goal of not needing to fully decode the video bitstream, not generating a reconstructed file, avoiding the risk of privacy leakage of the reconstructed file, and protecting the video privacy data to a certain extent. In addition, the scene detection network model based on the neural network model proposed in this application can avoid dependence on the original video data while realizing the prediction of the results in the target detection scenario, realizing the protection of video privacy and realizing the intelligence of home products. At the same time, in this method, this application will also store the latest transform domain coefficient data, and transmit the processed transform domain coefficient data to the cloud for subsequent equipment updates and function upgrades, etc., to solve the timeliness problem of smart home equipment. This application can support more scene applications, such as home monitoring, face recognition, etc.

[0099] By applying the technical solution of the present application, after the video data is collected by the camera of the smart home, the unreadable video transform domain coefficients of the video data are collected and the original video data is cleared, so that the home scene can be detected according to the video transform domain coefficient data and the learning model in the future. This avoids the problem of potential safety hazards caused by the original video data being readable after leakage.

[0100] Optionally, in another embodiment of the present application, Figure 5 As shown, the present application also provides a device for detecting home scenes. The device includes:

[0101] The first acquisition module 201 is configured to use a camera device provided in a smart home terminal to acquire home scene video data for a target area;

[0102] The second acquisition module 202 is configured to acquire target video transform domain coefficients of the home scene video data, and input the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area;

[0103] The execution module 203 is configured to control the smart home terminal to execute the action corresponding to the user operation event, if any.

[0104] In the present application, a camera device installed in a smart home terminal can be used to collect home scene video data for a target area; collect target video transform domain coefficients of the home scene video data, and input the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area; if so, control the smart home terminal to perform an action corresponding to the user operation event. By applying the technical solution of the present application, after the video data is collected by the camera of the smart home, the unreadable video transform domain coefficients of the video data are collected and the original video data is cleared, so that the home scene can be detected later based on the video transform domain coefficient data and the learning model. This avoids the problem of potential safety hazards caused by the readability of the original video data after leakage.

[0105] In another embodiment of the present application, the first acquisition module 201 is configured to perform the following steps:

[0106] Collecting transform domain coefficients of coding units CU of home scene video data; and, collecting transform domain coefficients of the largest coding unit LCU of home scene video data; and, collecting transform domain coefficients of single frames of home scene video data;

[0107] The CU transform domain coefficients, the LCU transform domain coefficients and the single frame transform domain coefficients are combined as the target video transform domain coefficients.

[0108] In another embodiment of the present application, the first acquisition module 201 is configured to perform the following steps:

[0109] Decoding the home scene video data to obtain decoded video data;

[0110] The target video transform domain coefficients of the decoded video data are acquired.

[0111] In another embodiment of the present application, the first acquisition module 201 is configured to perform the following steps:

[0112] The home scene video data is cleared.

[0113] In another embodiment of the present application, the first acquisition module 201 is configured to perform the following steps:

[0114] According to the identification of the target video transform domain coefficients, determining whether there are human features in the target area where the smart home terminal is currently located, the human features including at least one of size features, color features and contour features;

[0115] If it is determined that the human body feature exists, the target video transform domain coefficient is input into a preset scene detection network model to detect the user operation event.

[0116] In another embodiment of the present application, the first acquisition module 201 is configured to perform the following steps:

[0117] Recording the association relationship and associated data between the target video transform domain coefficient and the action of executing the user operation event;

[0118] After the association relationship and the association data are combined into model training data, the model training data is uploaded to the cloud, and the model training data is used to train the scene detection network model.

[0119] In another embodiment of the present application, the first acquisition module 201 is configured to perform the following steps:

[0120] After receiving the model update instruction, receiving the updated scene detection network model transmitted from the cloud; and clearing the scene detection network model originally stored in the smart home terminal.

[0121] Figure 63 is a logical structure block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0122] In an exemplary embodiment, a non-temporary computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by an electronic device processor to complete the above method of detecting home scenes, the method comprising: using a camera device set in a smart home terminal to collect home scene video data for a target area; collecting the target video transform domain coefficients of the home scene video data, and inputting the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area; if so, controlling the smart home terminal to perform the action corresponding to the user operation event. Optionally, the above instructions can also be executed by a processor of an electronic device to complete other steps involved in the above exemplary embodiments. For example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0123] In an exemplary embodiment, an application / computer program product is also provided, including one or more instructions, which can be executed by a processor of an electronic device to complete the above-mentioned method for detecting home scenes, the method comprising: using a camera device set in a smart home terminal to collect home scene video data for a target area; collecting target video transform domain coefficients of the home scene video data, and inputting the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area; if so, controlling the smart home terminal to perform an action corresponding to the user operation event. Optionally, the above-mentioned instructions can also be executed by a processor of an electronic device to complete other steps involved in the above-mentioned exemplary embodiment.

[0124] Those skilled in the art will understand that Figure 6 It is only an example of the electronic device 300 and does not constitute a limitation of the electronic device 300. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 300 may also include input and output devices, network access devices, buses, etc.

[0125] The processor 302 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor 302 may also be any conventional processor, etc. The processor 302 is the control center of the electronic device 300, and uses various interfaces and lines to connect various parts of the entire electronic device 300.

[0126] The memory 301 can be used to store computer-readable instructions. The processor 302 implements various functions of the electronic device 300 by running or executing the computer-readable instructions or modules stored in the memory 301 and calling the data stored in the memory 301. The memory 301 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the electronic device 300, etc. In addition, the memory 301 can include a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM) or other non-volatile / volatile storage devices.

[0127] If the module integrated in the electronic device 300 is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware to complete it through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when the computer-readable instructions are executed by the processor, the steps of the above-mentioned various method embodiments can be implemented.

[0128] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0129] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for detecting a home scene, characterized in that: The method is executed by a smart home terminal, comprising: Using a camera device installed in a smart home terminal to collect home scene video data for a target area; Collecting target video transform domain coefficients of the home scene video data, clearing the home scene video data, and inputting the target video transform domain coefficients into a compressed scene detection network model in a smart home terminal, wherein the compressed scene detection network model is obtained by compressing a preset scene detection network model, and the prediction result output by the compressed scene detection network model is a prediction result for specifically specifying the category and position of an object, and the output of the compressed scene detection network model is related to the probability of the object, the position of the object, and the category of the object, and by identifying the category and position of the object, determining whether there is a user operation event in the target area; If so, controlling the smart home terminal to execute an action corresponding to the user operation event; The target video transform domain coefficients for collecting the home scene video data include: Collecting transform domain coefficients of coding units CU of home scene video data; and, collecting transform domain coefficients of the largest coding unit LCU of home scene video data; and, collecting transform domain coefficients of single frames of home scene video data; The CU transform domain coefficients, the LCU transform domain coefficients and the single frame transform domain coefficients are combined as the target video transform domain coefficients.

2. The method according to claim 1, characterized in that The target video transform domain coefficients for collecting the home scene video data include: Decoding the home scene video data to obtain decoded video data; The target video transform domain coefficients of the decoded video data are acquired.

3. The method according to claim 1, characterized in that The step of inputting the target video transform domain coefficients into a preset scene detection network model to detect whether there is a user operation event in the target area includes: According to the identification of the target video transform domain coefficients, determining whether there are human features in the target area where the smart home terminal is currently located, the human features including at least one of size features, color features and contour features; If it is determined that the human body feature exists, the target video transform domain coefficient is input into a preset scene detection network model to detect the user operation event.

4. The method according to claim 1, characterized in that After controlling the smart home terminal to execute the action corresponding to the user operation event, the method further includes: Recording the association relationship and associated data between the target video transform domain coefficient and the action of executing the user operation event; After the association relationship and the association data are combined into model training data, the model training data is uploaded to the cloud, and the model training data is used to train the scene detection network model.

5. The method according to claim 4, characterized in that After merging the association relationship and the association data into model training data and uploading the model training data to the cloud, the method further includes: After receiving the model update instruction, receiving the updated scene detection network model transmitted from the cloud; and clearing the scene detection network model originally stored in the smart home terminal.

6. A device for detecting home scenes, characterized in that: The device is executed by a smart home terminal, including: The first acquisition module is configured to use a camera device provided in a smart home terminal to acquire home scene video data for a target area; The second acquisition module is configured to acquire the target video transform domain coefficients of the home scene video data, clear the home scene video data, and input the target video transform domain coefficients into the compressed scene detection network model in the smart home terminal, wherein the compressed scene detection network model is compressed by a preset scene detection network model, and the prediction result output by the compressed scene detection network model is a prediction result for specifically specifying the category and position of an object, and the output of the compressed scene detection network model is related to the probability of the object, the position of the object and the category of the object, and by identifying the category and position of the object, it is determined whether there is a user operation event in the target area; wherein, acquiring the target video transform domain coefficients of the home scene video data includes: acquiring the transform domain coefficients of the coding unit CU of the home scene video data; and acquiring the transform domain coefficients of the largest coding unit LCU of the home scene video data; and acquiring the transform domain coefficients of the single frame of the home scene video data; and merging the CU transform domain coefficients, the LCU transform domain coefficients and the single frame transform domain coefficients as the target video transform domain coefficients; The execution module is configured to control the smart home terminal to execute the action corresponding to the user operation event, if it exists.

7. An electronic device, characterized in that: include: A memory for storing executable instructions; as well as, A processor is used to execute the executable instructions with the memory to complete the operation of the method for detecting a home scene as described in any one of claims 1-5.

8. A computer-readable storage medium for storing computer-readable instructions, characterized in that: When the instruction is executed, the operation of the method for detecting home scenes described in any one of claims 1-5 is performed.

Images