Smart home and control method, module and computer readable medium thereof

Abstract

The present application relates to a smart home and its control method, module, and computer-readable medium. The method includes: obtaining multimedia data collected from a first object, the first object is located within a first spatial range, and a first smart home device is installed within the first spatial range; inputting the multimedia data into a target detection network model to extract target features of the first object using the target detection network model; when the target features are consistent with the features extracted from a second object within a second spatial range, determining that the first object and the second object are the same object, and setting the operating parameters of the first smart home device according to the operating parameters of the second smart home device installed within the second spatial range. The present application solves the technical problem that it is difficult for users to synchronize personalized settings of smart home devices to multiple spaces.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and in particular to a smart home and a control method, module, and computer-readable medium thereof. Background Art

[0002] With the rapid development of smart homes, the demand for "intelligence" is no longer limited to one space. More people hope that personalized settings can be applied more intelligently to multiple scenarios.

[0003] At present, in the relevant technologies, the effective range of "intelligence" is affected by factors such as distance and signal strength, and the user's personalized settings are difficult to synchronize to multiple spaces.

[0004] Currently, there is no effective solution to the problem that users' personalized settings for smart home devices are difficult to synchronize to multiple spaces. Summary of the invention

[0005] The present application provides a smart home and its control method, module, and computer-readable medium to solve the technical problem that it is difficult for users to synchronize personalized settings of smart home devices to multiple spaces.

[0006] According to one aspect of an embodiment of the present application, the present application provides a smart home control method, including: acquiring multimedia data collected from a first object, the first object is located within a first spatial range, and a first smart home device is installed within the first spatial range; inputting the multimedia data into a target detection network model to extract target features of the first object using the target detection network model; when the target features are consistent with the features extracted from a second object within a second spatial range, determining that the first object and the second object are the same object, and setting the operating parameters of the first smart home device according to the operating parameters of the second smart home device installed within the second spatial range.

[0007] Optionally, inputting multimedia data into a target detection network model to extract target features of a first object using the target detection network model includes: inputting a video sequence into multiple dense convolution blocks, and obtaining a first frame-level feature of a convolution layer output corresponding to each dense convolution block, wherein any two dense convolution blocks have different sizes, and the multimedia data includes a video sequence; and fusing the first frame-level features output by all convolution layers to obtain the target feature.

[0008] Optionally, after the video sequence is input into multiple dense convolution blocks, the method also includes: in the dense convolution blocks, dividing the video sequence into multiple video frames; performing image recognition on the video frames of the video sequence frame by frame to extract at least one of the clothing, headdress, and body posture of the first object as a first frame-level feature.

[0009] Optionally, inputting multimedia data into a target detection network model to extract target features of a first object using the target detection network model also includes: inputting a speech sequence into multiple dense convolution blocks, and obtaining second frame-level features of the convolution layer output corresponding to each dense convolution block, wherein the multimedia data includes a speech sequence; and fusing the second frame-level features output by all convolution layers to obtain the target features.

[0010] Optionally, after the speech sequence is input into multiple dense convolution blocks, the method also includes: in the dense convolution blocks, dividing the speech sequence into multiple speech frames; performing voiceprint recognition on the speech frames of the speech sequence frame by frame to extract the voiceprint features of the first object as the second frame-level features.

[0011] Optionally, the first frame-level features output by all convolutional layers are fused to obtain the target features, including: assigning a weight parameter to each first frame-level feature, and determining a weighted average of all first frame-level features to obtain a sequence-level target feature; the second frame-level features output by all convolutional layers are fused to obtain the target features, including: assigning a weight parameter to each second frame-level feature, and determining a weighted average of all second frame-level features to obtain a sequence-level target feature.

[0012] Optionally, the method further includes: when all the saved object features are inconsistent with the target feature, determining that the first object is an unknown object, and sending an unknown object identity confirmation prompt message to the audio playback device and / or the display device.

[0013] According to another aspect of an embodiment of the present application, the present application provides a control module of a smart home, including: a data acquisition module, used to acquire multimedia data collected from a first object, the first object is located within a first spatial range, and a first smart home device is installed within the first spatial range; a feature extraction module, used to input the multimedia data into a target detection network model to extract target features of the first object using the target detection network model; an operating parameter migration module, used to determine that the first object and the second object are the same object when the target features are consistent with the features extracted from the second object within the second spatial range, and to set the operating parameters of the first smart home device according to the operating parameters of the second smart home device installed within the second spatial range.

[0014] According to another aspect of an embodiment of the present application, the present application provides a smart home, including a memory, a processor, a communication interface and a communication bus, the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the steps of the above method are implemented when the processor executes the computer program.

[0015] According to another aspect of an embodiment of the present application, the present application also provides a computer-readable medium having a non-volatile program code executable by a processor, and the program code enables the processor to execute the above method.

[0016] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:

[0017] The technical solution of the present application is to obtain multimedia data collected for a first object, the first object is located within a first spatial range, and a first smart home device is installed within the first spatial range; the multimedia data is input into a target detection network model to use the target detection network model to extract target features of the first object; when the target features are consistent with the features extracted for the second object within a second spatial range, it is determined that the first object and the second object are the same object, and the operating parameters of the first smart home device are set according to the operating parameters of the second smart home device installed within the second spatial range. When it is necessary to set parameters for the first smart home device within the first spatial range, the present application compares user features collected from different spaces. If a second spatial range with the same features is found, the first smart home device is adjusted according to the parameter settings of the second smart home device within the second spatial range by the user, thereby solving the technical problem that the personalized settings of the user for the smart home device are difficult to synchronize to multiple spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 A schematic diagram of a hardware environment of an optional smart home control method provided according to an embodiment of the present application;

[0021] Figure 2 A flowchart of an optional smart home control method provided according to an embodiment of the present application;

[0022] Figure 3 A schematic diagram of an optional space and smart home provided according to an embodiment of the present application;

[0023] Figure 4 A schematic diagram of an optional target detection network model provided according to an embodiment of the present application;

[0024] Figure 5 A block diagram of an optional smart home control module provided according to an embodiment of the present application;

[0025] Figure 6 An optional smart home structure schematic diagram provided for an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0027] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0028] In order to solve the problem mentioned in the background technology, according to one aspect of the embodiments of the present application, an embodiment of a method is provided.

[0029] Optionally, in the embodiment of the present application, the above method can be applied to Figure 1 In the hardware environment composed of the terminal 101 and the server 103 shown in FIG. Figure 1 As shown, the server 103 is connected to the terminal 101 via a network, and can be used to provide services for the terminal or a client installed on the terminal. A database 105 can be set on the server or independently of the server to provide data storage services for the server 103. The above-mentioned network includes but is not limited to: a wide area network, a metropolitan area network or a local area network, and the terminal 101 includes but is not limited to a PC, a mobile phone, a tablet computer, etc.

[0030] A method in the embodiment of the present application can be executed by the server 103, or can be executed by the server 103 and the terminal 101 together, such as Figure 2 As shown, the method may include the following steps:

[0031] Step S202, acquiring multimedia data collected from a first object, where the first object is located within a first spatial range, and a first smart home device is installed within the first spatial range;

[0032] Step S204, inputting the multimedia data into the target detection network model to extract target features of the first object using the target detection network model;

[0033] Step S206, when the target feature is consistent with the feature extracted from the second object within the second spatial range, it is determined that the first object and the second object are the same object, and the operating parameters of the first smart home device are set according to the operating parameters of the second smart home device installed within the second spatial range.

[0034] Through the above steps S202 to S206, when it is necessary to set parameters for the first smart home device in the first spatial range, the present application compares the user characteristics collected from different spaces. If a second spatial range with the same characteristics is found, the first smart home device is adjusted according to the parameter settings of the second smart home device in the second spatial range by the user, thereby solving the technical problem that the user's personalized settings of the smart home device are difficult to synchronize to multiple spaces.

[0035] In the present application embodiment, Figure 3 As shown, there are independent rooms in an independent and complete physical space, and each room includes a smart home device with the same function, that is, the first smart home device in the first space range and the second smart home device in the second space range. The smart home device can be an air conditioner, a fan, etc. Users can move freely in each room. Taking the air conditioner as an example, the air conditioner installed in each room may include a camera for detecting the identity of the user in the room detection area and obtaining the user's identity information; a control storage unit for realizing the storage, matching and other operations of the user's identity information, as well as the transmission of control instructions, and the stored user identity information realizes information interconnection through the network; a temperature and wind speed adjustment unit for realizing the regulation of temperature and air supply parameters; a voice playback unit for realizing the playback of voice and music. It can also include a recording device for collecting the user's voice; a display device for displaying the temperature, wind speed, time, prompt information, etc. of the air conditioner.

[0036] Optionally, step S204 of inputting multimedia data into the target detection network model to extract target features of the first object using the target detection network model may include:

[0037] Step 11: input the video sequence into multiple dense convolution blocks, and obtain the first frame-level features of the convolution layer output corresponding to each dense convolution block, wherein the sizes of any two dense convolution blocks are different, and the multimedia data includes the video sequence.

[0038] In an embodiment of the present application, after the video sequence is input into multiple dense convolution blocks, the video sequence can be divided into multiple video frames in the dense convolution blocks; image recognition is performed on the video frames of the video sequence frame by frame to extract at least one of the clothing, headdress, and body posture of the first object as a first frame-level feature.

[0039] Step 12: fuse the first frame-level features output by all convolutional layers to obtain the target features.

[0040] In the embodiment of the present application, a video of the first spatial range can be captured by a camera as the above-mentioned video sequence, and feature extraction is performed when the video sequence is input into the target detection network model, wherein, Figure 4 As shown, the target detection network model can be to replace the residual layer in the YOLOv3 network with a dense convolution block, construct a feature pyramid containing 5 convolution layers of different scales, fuse the feature pyramid with the deep dense network, and use the Softmax function to classify the fused features to form a deep fusion fast target detection model. The different scales of the above-mentioned multiple dense convolution blocks can be different resolutions of the dense convolution blocks. Dense convolution blocks of different resolutions can obtain feature maps of different sizes, so that features of different scales of the video sequence can be obtained.

[0041] When performing feature extraction, the video sequence can be subjected to image recognition frame by frame, thereby extracting feature information such as clothing, headwear, and body shape of the first object in each frame, obtaining the above-mentioned first-frame-level features of each frame, and then merging the first-frame-level features of all frames to obtain sequence-level features. Finally, the Softmax function is used to minimize the loss and obtain the optimal recognition result.

[0042] In the embodiment of the present application, clothing may include the style, type, color, etc. of clothing, headwear may include hats, hairstyles, etc., and body shape may include posture, appearance such as height, weight, etc.

[0043] Optionally, fusing the first frame-level features output by all convolutional layers to obtain the target features includes: assigning a weight parameter to each first frame-level feature, and determining a weighted average of all first frame-level features to obtain a sequence-level target feature.

[0044] In the embodiment of the present application, a regularization constraint is added during feature extraction, that is, a weight parameter is assigned to each first frame-level feature to avoid the weight parameter from being aggregated in one frame. During feature fusion, an attention mechanism is used to weight the average of the frame-level features, so that the first frame-level features of all frames are fused to obtain a sequence-level target feature.

[0045] By adopting the technical solution of this application, artificial intelligence can accurately track the position of people across cameras and scenes through information such as clothing, hairstyle, and body shape. Computer vision technology can be used to determine whether there is a specific user in the first spatial range (that is, the user who sets personalized parameters for the second smart home device in the second spatial range). The user can be identified based on clothing, body shape, hairstyle, etc. without recognizing the face. The personalized parameters set by the user in the second spatial range for the second smart home device can then be synchronized to the first smart home device in the first spatial range, further embodying the concept of smart home and improving user experience.

[0046] In the embodiments of the present application, not only can computer vision technology be used to extract features from the user's video sequence, but voiceprint recognition technology can also be used to extract features from the user's voice.

[0047] Optionally, inputting the multimedia data into the target detection network model to extract the target feature of the first object using the target detection network model may also include:

[0048] Step 21, inputting the speech sequence into a plurality of dense convolution blocks, and obtaining a second frame-level feature of the convolution layer output corresponding to each dense convolution block, wherein the multimedia data includes the speech sequence.

[0049] In an embodiment of the present application, after a speech sequence is input into a plurality of dense convolution blocks, the speech sequence can be divided into a plurality of speech frames in the dense convolution blocks; voiceprint recognition is performed on the speech frames of the speech sequence frame by frame to extract the voiceprint features of the first object as the second frame-level features.

[0050] Step 22, the second frame level features output by all convolutional layers are fused to obtain the target features.

[0051] In an embodiment of the present application, when performing feature extraction, the speech sequence can be identified frame by frame, so as to extract the voiceprint feature of the first object in each frame as the above-mentioned second frame-level feature, and then the second frame-level features of all speech frames are fused to obtain sequence-level features, and finally the Softmax function is used to minimize the loss to obtain the optimal recognition result.

[0052] In the embodiment of the present application, the above-mentioned voiceprint features may include: acoustic features (cephalogram); lexical features (speaker-related words, phonemes); prosodic features; language, dialect and accent information; channel information, etc.

[0053] Optionally, the second frame-level features output by all convolutional layers are fused to obtain the target features, including: assigning a weight parameter to each second frame-level feature, and determining a weighted average of all second frame-level features to obtain a sequence-level target feature.

[0054] In the embodiment of the present application, a regularization constraint is added during feature extraction, that is, a weight parameter is assigned to each second frame-level feature to avoid the weight parameter from being aggregated in one frame. During feature fusion, an attention mechanism is used to weight the average of the frame-level features, so that the second frame-level features of all speech frames are fused to obtain a sequence-level target feature.

[0055] By adopting the technical solution of this application, artificial intelligence can accurately track the location of people across space and scenes through the user's voiceprint information. By using voiceprint recognition technology to extract features from the voice sequence, it is possible to determine whether there is a specific user in the first spatial range (that is, the user who has set personalized parameters for the second smart home device in the second spatial range). The user can be identified based on voice alone without recognizing the face, and the personalized parameters set by the user in the second spatial range for the second smart home device can be synchronized to the first smart home device in the first spatial range, further embodying the concept of smart home and improving user experience.

[0056] In the embodiment of the present application, a security function is also provided for prompting strangers after entering the first space range.

[0057] Optionally, the method further includes: when all the saved object features are inconsistent with the target feature, determining that the first object is an unknown object, and sending an unknown object identity confirmation prompt message to the audio playback device and / or the display device.

[0058] In the embodiment of the present application, if the first object is a stranger, the first smart home device sends an unknown object identity confirmation prompt message through the audio playback device and / or the display device to promptly remind the homeowner to confirm the identity of the stranger. The technical solution of the present application can prevent strangers from breaking in to a certain extent and improve the user experience.

[0059] According to another aspect of the embodiment of the present application, Figure 5 As shown, a control module of a smart home is provided, comprising:

[0060] The data acquisition module 501 is used to acquire multimedia data collected from a first object, where the first object is located within a first spatial range, and a first smart home device is installed within the first spatial range;

[0061] A feature extraction module 503, used for inputting multimedia data into the target detection network model to extract target features of the first object using the target detection network model;

[0062] The operating parameter migration module 505 is used to determine that the first object and the second object are the same object when the target feature is consistent with the feature extracted from the second object within the second spatial range, and set the operating parameters of the first smart home device according to the operating parameters of the second smart home device installed within the second spatial range.

[0063] In the embodiment of the present application, the data acquisition module 501 acquires multimedia data of a first object located within a first spatial range, the feature extraction module 503 inputs the multimedia data into a target detection model to extract features of the first object, and the running parameter migration module 505 determines that the first object and the second object are the same object when the features of the first object are consistent with the features of the second object, and migrates the parameters set for the second smart home device by the second object within the second spatial range to the first smart home device within the first spatial range, so that when it is necessary to set parameters for the first smart home device within the first spatial range, by comparing user features collected from different spaces, if a second spatial range with the same features is found, the first smart home device is adjusted according to the parameter settings of the user for the second smart home device within the second spatial range, thereby solving the technical problem that the user's personalized settings for smart home devices are difficult to synchronize to multiple spaces.

[0064] It should be noted that the data acquisition module 501 in this embodiment can be used to execute step S202 in the embodiment of the present application, the feature extraction module 503 in this embodiment can be used to execute step S204 in the embodiment of the present application, and the operating parameter migration module 505 in this embodiment can be used to execute step S206 in the embodiment of the present application.

[0065] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. Figure 1 In the hardware environment shown, it can be implemented by software or by hardware.

[0066] Optionally, the feature extraction module also includes an image feature extraction unit, which is specifically used to: input the video sequence into multiple dense convolution blocks, and obtain the first frame-level features of the convolution layer output corresponding to each dense convolution block, wherein the sizes of any two dense convolution blocks are different, and the multimedia data includes the video sequence; fuse the first frame-level features output by all convolution layers to obtain the target features.

[0067] Optionally, the image feature extraction unit is further used to: divide the video sequence into multiple video frames in a dense convolution block; perform image recognition on the video frames of the video sequence frame by frame to extract at least one of the clothing, headdress, and body posture of the first object as a first frame-level feature.

[0068] Optionally, the feature extraction module also includes a voiceprint feature extraction unit, which is specifically used to: input the speech sequence into multiple dense convolution blocks, and obtain the second frame-level features of the convolution layer output corresponding to each dense convolution block, wherein the multimedia data includes the speech sequence; and fuse the second frame-level features output by all convolution layers to obtain the target features.

[0069] Optionally, the voiceprint feature extraction unit is further used to: divide the speech sequence into multiple speech frames in a dense convolution block; perform voiceprint recognition on the speech frames of the speech sequence frame by frame to extract the voiceprint feature of the first object as the second frame-level feature.

[0070] Optionally, the image feature extraction unit is further used to: assign a weight parameter to each first frame level feature, and determine a weighted average of all first frame level features to obtain a sequence level target feature.

[0071] Optionally, the voiceprint feature extraction unit is further used to: assign a weight parameter to each second frame level feature, and determine a weighted average of all second frame level features to obtain a sequence level target feature.

[0072] Optionally, the control module of the smart home also includes a security module, which is used to: when all saved object features are inconsistent with the target features, determine that the first object is an unknown object, and send an unknown object identity confirmation prompt message to the audio playback device and / or display device.

[0073] According to another aspect of the embodiment of the present application, the present application provides a smart home, such as Figure 6 As shown, it includes a memory 601, a processor 603, a communication interface 605 and a communication bus 607. The memory 601 stores a computer program that can be run on the processor 603. The memory 601 and the processor 603 communicate through the communication interface 605 and the communication bus 607. When the processor 603 executes the computer program, the steps of the above method are implemented.

[0074] The memory and processor in the above-mentioned smart home communicate through a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0075] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0076] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0077] According to another aspect of the embodiments of the present application, a computer-readable medium having a non-volatile program code executable by a processor is provided.

[0078] Optionally, in an embodiment of the present application, the computer-readable medium is configured to store program codes for the processor to perform the following steps:

[0079] Acquire multimedia data collected from a first object, where the first object is located within a first spatial range, and a first smart home device is installed within the first spatial range;

[0080] Inputting the multimedia data into the target detection network model to extract target features of the first object using the target detection network model;

[0081] When the target feature is consistent with the feature extracted from the second object within the second spatial range, it is determined that the first object and the second object are the same object, and the operating parameters of the first smart home device are set according to the operating parameters of the second smart home device installed within the second spatial range.

[0082] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0083] When the embodiments of the present application are specifically implemented, reference may be made to the above-mentioned embodiments, which have corresponding technical effects.

[0084] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0085] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0086] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0087] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0088] In the embodiments provided in the present application, it should be understood that the disclosed modules and methods can be implemented in other ways. For example, the module embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, modules or units, which can be electrical, mechanical or other forms.

[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0091] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0092] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A control method for a smart home, characterized in that: include: Acquire multimedia data collected from a first object, wherein the first object is located within a first spatial range, and a first smart home device is installed within the first spatial range; Inputting the multimedia data into a target detection network model to extract target features of the first object using the target detection network model, wherein the target detection network model is based on a YOLOv3 network, and a residual layer is replaced by a dense convolution block, and a plurality of the dense convolution blocks have different scales to extract features of different scales of the multimedia data, and the plurality of the dense convolution blocks are used to construct a feature pyramid, and the feature pyramid is fused with a deep dense network to classify the fused features using a Softmax function; When the target feature is consistent with the feature extracted from the second object within the second spatial range, it is determined that the first object and the second object are the same object, and the operating parameters of the first smart home device are set according to the operating parameters of the second smart home device installed within the second spatial range.

2. The method according to claim 1, characterized in that Inputting the multimedia data into a target detection network model to extract target features of the first object using the target detection network model includes: Inputting a video sequence into a plurality of dense convolution blocks, and obtaining a first frame-level feature of a convolution layer output corresponding to each of the dense convolution blocks, wherein any two of the dense convolution blocks have different sizes, and the multimedia data includes the video sequence; The first frame-level features output by all the convolutional layers are fused to obtain the target features.

3. The method according to claim 2, characterized in that After inputting the video sequence into a plurality of dense convolution blocks, the method further comprises: In the dense convolution block, the video sequence is divided into a plurality of video frames; Image recognition is performed on the video frames of the video sequence frame by frame to extract at least one of clothing, headwear, and body posture of the first object as the first frame-level feature.

4. The method according to claim 2, characterized in that: Inputting the multimedia data into a target detection network model to extract target features of the first object using the target detection network model further includes: Inputting a speech sequence into a plurality of the dense convolution blocks, and obtaining a second frame-level feature of a convolution layer output corresponding to each of the dense convolution blocks, wherein the multimedia data includes the speech sequence; The second frame-level features output by all the convolutional layers are fused to obtain the target features.

5. The method according to claim 4, characterized in that After inputting the speech sequence into a plurality of the dense convolution blocks, the method further comprises: In the dense convolution block, the speech sequence is divided into a plurality of speech frames; Voiceprint recognition is performed on the speech frames of the speech sequence frame by frame to extract the voiceprint features of the first object as the second frame-level features.

6. The method according to claim 4, characterized in that Fusing the first frame-level features output by all the convolutional layers to obtain the target feature comprises: assigning a weight parameter to each of the first frame-level features, and determining a weighted average of all the first frame-level features to obtain the target feature at the sequence level; Fusing the second frame-level features output by all the convolutional layers to obtain the target feature includes: assigning a weight parameter to each of the second frame-level features, and determining a weighted average of all the second frame-level features to obtain the target feature at the sequence level.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When all the saved object features are inconsistent with the target feature, the first object is determined to be an unknown object, and an unknown object identity confirmation prompt message is sent to the audio playback device and / or the display device.

8. A control module for a smart home, characterized in that: include: A data acquisition module, configured to acquire multimedia data collected from a first object, wherein the first object is located within a first spatial range, and a first smart home device is installed within the first spatial range; A feature extraction module, used for inputting the multimedia data into a target detection network model, so as to extract the target features of the first object by using the target detection network model, wherein the target detection network model is formed by replacing the residual layer with a dense convolution block based on a YOLOv3 network, and the scales of the plurality of dense convolution blocks are different, so as to extract features of different scales of the multimedia data, and the plurality of dense convolution blocks are used for constructing a feature pyramid, and the feature pyramid is fused with a deep dense network, so as to classify the fused features by using a Softmax function; An operating parameter migration module is used to determine that the first object and the second object are the same object when the target feature is consistent with the feature extracted from the second object within the second spatial range, and to set the operating parameters of the first smart home device according to the operating parameters of the second smart home device installed within the second spatial range.

9. A smart home, comprising a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, and the memory and the processor communicate through the communication bus and the communication interface, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer readable medium having a non-volatile program code executable by a processor, characterized in that: The program code enables the processor to execute the method according to any one of claims 1 to 7.

Images