Speaker control management method and system based on data analysis

Abstract

The present invention discloses a speaker control management method and system based on data analysis, which belongs to the field of intelligent speaker control technology. The system includes a data acquisition module, an environment analysis module, an operation management module and an automatic storage module; the data acquisition module is used to collect a voiceprint feature library and historical records, as well as audio information and image information in a specified area; the environment analysis module determines whether an object exists through audio and image, establishes a visual area for the object, and calculates a trigger index according to the change in sound intensity when an intelligent speaker exists in the visual area and determines whether to trigger the intelligent speaker; the operation management module performs voiceprint recognition on the audio, obtains historical records corresponding to the voiceprint features, executes instructions according to the historical records and enters an interactive state, calculates a state index according to the time difference of the instructions, and thus sets the state of the intelligent speaker; the automatic storage module is used to store the voiceprint features and sentence information in the interactive process in the voiceprint feature library and historical records respectively.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent speaker control, and in particular to a speaker control management method and system based on data analysis. Background Art

[0002] With the rapid development of artificial intelligence and smart home technology, smart speakers as a smart device have become an indispensable part of people's lives. In order to improve the user experience and convenience of smart speakers, researchers began to explore how to activate smart speakers through more efficient wake-up methods to achieve voice control and interaction.

[0003] At present, smart speakers usually use pre-set wake-up words for voice wake-up, which has some disadvantages. For example: 1. In some cases, people may not be able to activate the smart speaker normally because they forget the wake-up word in a short time, and the degree of intelligence is not high. 2. The setting of some wake-up functions requires the smart speaker to convert voice into text in real time to determine whether to wake up and recognize instructions. These text contents may be illegally obtained and used by other applications, and even cause information leakage. 3. When people interact with smart speakers by voice, they often need to add wake-up words to the conversation every time to give instructions to the smart speaker, which is not in line with people’s normal conversation habits. Therefore, at this stage, a more efficient and intelligent smart speaker wake-up word-free activation technology solution is needed to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide a speaker control management method and system based on data analysis to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solution: a speaker control management method based on data analysis, the method comprising the following steps:

[0006] S100 collects the voiceprint feature library and historical records stored in the smart speaker, and uses different sensors to collect audio and image information in a specified area in real time, and comprehensively analyzes the audio and image to determine whether an object exists.

[0007] S200. Analyze the sight range of the object through the head posture detection algorithm to establish a visual area, mark the object when there is a smart speaker in the visual area, calculate the trigger index according to the change in sound intensity and determine whether to trigger the smart speaker.

[0008] S300, the smart speaker recognizes the voiceprint features in the audio, obtains the corresponding historical records, executes the corresponding instructions according to the historical records and enters the interactive state, calculates the state index through the instruction time difference and sets the smart speaker state.

[0009] S400: storing the voiceprint features and sentence information during the interaction process into a voiceprint feature library and a historical record respectively.

[0010] In S100, the voiceprint feature library refers to a database that stores voiceprint features. Each user corresponds to a database that stores different voiceprint features. The historical record refers to the user's command phrase record. Each user contains different command phrase records, and each command phrase record includes commands and phrases. The designated area refers to the area where the smart speaker is used. The camera collects image information in the designated area. The image is a real-time video, and the microphone collects audio information in the designated area. The steps to determine whether an object exists are as follows:

[0011] S101, the microphone collects the audio in the specified area in real time and calculates the sound intensity, and records the time when the sound intensity is greater than the intensity threshold , automatically controls the camera to start recording video in the specified area.

[0012] S102, using OpenCV technology to decompose the video into single-frame images, screening all single-frame images, using edge detection method to remove blurred single-frame images; using YOLOv5 human body detection algorithm to perform target recognition on the remaining single-frame images, and taking the recognized human targets as objects. If no object is recognized in the specified area, the camera is controlled to be turned off.

[0013] A command is an operation instruction that can be directly recognized and executed by a smart speaker. A term is a textual expression made by a user when issuing a command, which is obtained by converting the voice in the collected audio information into text. Each command corresponds to several terms, which can represent the same command, and different terms have different sentence structures or adjectives.

[0014] In S200, the specific steps are as follows:

[0015] S201, using a key point detection algorithm to identify different parts of the object and mark the key points, and obtaining the head deflection angle by analyzing the changes in the positions of these key points, and then combining the head posture detection algorithm to analyze and obtain the object's line of sight direction.

[0016] Each part of the object has a different number of key points, and key points are usually set at different positions of the main parts of the human body. The head deflection angle and line of sight direction are analyzed based on the position or angle changes of these key points.

[0017] S202: Obtaining the position coordinates of the object , set the horizontal viewing angle R and sensing distance L, with the position coordinates With the sensor distance L as the center and the radius as the radius, a sector area with an angle of R is divided in the specified area in the direction of sight as the object's visual area. A line segment with a distance of L is intercepted from the object's position along the direction of sight as the visual line. The visual line is the middle line of the visual area, and the visual area changes dynamically as the object moves or the head deflects.

[0018] The horizontal viewing angle and sensing distance are set in advance by the administrator. The specific values ​​are based on the attention angle of the human eye in the horizontal dimension and the anti-miscontrol distance of the smart speaker.

[0019] The attention angle of the human eye in the horizontal dimension refers to the range that the human visual system can focus and pay attention to in the horizontal direction. This angle is usually used to describe the range that the human eye can cover in the horizontal direction, that is, the area that the human eye can observe and pay attention to through eye movement and adjustment without turning the head.

[0020] The anti-miscontrol distance of smart speakers refers to the effective range within which the device can recognize and respond to human commands when using smart speakers. This distance usually refers to the distance between a person and the smart speaker. Beyond this distance, the smart speaker may not be able to accurately recognize user commands or respond effectively.

[0021] S203: Obtain the location coordinates of the smart speaker in the specified area , real-time analysis of the dynamic changes in the visual area of ​​the object, the visual area contains the position coordinates Then mark the object and record the time ,according to arrive The average sound intensity is calculated from the audio information during this period .

[0022] The visual area of ​​the marker object always contains the position coordinates , when the visual area no longer contains the position coordinates , the object is unmarked.

[0023] S204: Use the line segment connecting the position of the marked object and the position of the smart speaker as the baseline, and obtain the angle between the visual line and the baseline , and time The sound intensity after , set the sound intensity increase value , calculate the trigger index of the marked object at different times in real time, when the trigger index is greater than the index threshold The smart speaker state is defined as triggered when, the formula is as follows:

[0024] ;

[0025] In the formula, is the trigger index, is the baseline length, and is a constant.

[0026] When the baseline length is smaller, the angle The smaller the sound or the higher the sound intensity, the greater the trigger index. The trigger index is used to quantify the likelihood that the subject is ready to use the smart speaker.

[0027] In S300, the specific steps are as follows:

[0028] S301: When the state of the smart speaker is triggered, the voiceprint recognition algorithm is used to extract the time The voiceprint characteristics of the subsequent audio , the voiceprint feature Match all the voiceprint features in the voiceprint feature library and calculate the similarity. Sum the similarities of all voiceprint features in the same database and divide it by the total number of voiceprint features in the database to get the average similarity. Mark the database with the highest average similarity. When the average similarity of the marked database is greater than the similarity threshold When , the smart speaker state is defined as interaction, and the user corresponding to the database is marked as the associated user.

[0029] Voiceprint feature matching uses text-independent voiceprint recognition, which refers to the use of individual voice features for identity verification or identification without relying on specific text content. Voiceprint recognition is a biometric technology that identifies people by analyzing the tone, timbre or speaking speed of voice features.

[0030] S302: When the state of the smart speaker is interactive, a term set is established, and the terms in all the command term records corresponding to the associated user in the historical records are put into the term set, and the time is The audio is then converted into text. , each term and sentence in the term set Calculate the text similarity separately, mark the terms with the highest text similarity, and when the text similarity of the marked terms is greater than the similarity threshold Record time , the smart speaker executes the instructions corresponding to the marked terms.

[0031] S303, continuously analyzing the speech in the audio information, substituting it into step S302 to calculate the text similarity and update the time ; Get the current time Substitute it into the formula to calculate the state index of the smart speaker at different times:

[0032] ;

[0033] In the formula, is the state index, is a constant. Set the state index threshold. When the state index is less than the state index, the smart speaker state is defined as triggered. When the state index is not greater than zero, all state definitions of the smart speaker are cancelled. When the visual area does not contain the position coordinates , unmarks the object.

[0034] The state of the smart speaker is set by the state index. In the interactive state, the smart speaker no longer performs voiceprint recognition, but only performs voice-to-text conversion, recognizes and executes commands. In the triggered state, the smart speaker no longer performs voice-to-text conversion, recognizes and executes commands, but only performs voiceprint recognition to determine whether to switch states. In the none state, only the trigger index of the marked object is calculated to determine whether to set the smart speaker to the triggered state. The visual area does not contain position coordinates Automatically unmark objects when

[0035] In S400, the voiceprint features during the interaction between the marked object and the smart speaker are stored in the database of the associated user in the voiceprint feature library, and the sentence information and the corresponding instructions are recorded as the instruction terms of the associated user and stored in the historical record.

[0036] The speaker control and management system based on data analysis includes a data acquisition module, an environment analysis module, an operation management module and an automatic storage module.

[0037] The data acquisition module is used to collect voiceprint feature libraries and historical records, as well as audio and image information in the specified area. The environmental analysis module determines whether there is an object through audio and image, establishes a visual area for the object, and calculates the trigger index based on the change in sound intensity when there is a smart speaker in the visual area and determines whether to trigger the smart speaker. The operation management module performs voiceprint recognition on the audio, obtains the historical records corresponding to the voiceprint features, executes instructions according to the historical records and enters the interactive state, calculates the state index through the time difference of the instructions, and thus sets the state of the smart speaker. The automatic storage module is used to store information in the interactive process in real time.

[0038] The data collection module includes a feature library collection unit, a history record collection unit, an image information collection unit and an audio information collection unit.

[0039] The feature library acquisition unit is used to collect and store the database of voiceprint features. Each user corresponds to a database, and different voiceprint features are stored in the database. The history record acquisition unit is used to collect the command terminology records of each user. Each command terminology record includes the command and terminology. The image information acquisition unit collects the image information in the specified area through the camera, and the image is real-time video. The audio information acquisition unit collects the audio information in the specified area through the microphone.

[0040] The environment analysis module includes an object analysis unit and a wake-up judgment unit.

[0041] The object analysis unit is used to identify objects in a specified area. The sound intensity is calculated through audio information. If the sound intensity is greater than the intensity threshold, the time is recorded. The camera is controlled to record the video. The video is deframed into single-frame images, and after the clear single-frame images are screened out, the human body detection algorithm is used for target recognition, and the recognized human body target is regarded as the object.

[0042] The wake-up judgment unit is used to judge whether to wake up the smart speaker.

[0043] First, the key point detection algorithm is used to mark the key points of different parts of the object, and the line of sight of the object is obtained based on the changes in the positions of these key points.

[0044] Secondly, set the horizontal viewing angle R and sensing distance L to the object's position coordinates. As the center of the circle, L is the radius, and a sector area with an angle of R is divided in the specified area towards the line of sight as the visual area of ​​the object, and the middle line of the visual area is the visual line; when the visual area contains the position coordinates of the smart speaker When the object is marked and the time is recorded ,calculate arrive The average sound intensity during this period .

[0045] Finally, the line segment connecting the position of the marked object and the position of the smart speaker is used as the baseline to obtain the angle between the visual line and the baseline. , and time The sound intensity after , set the sound intensity increase value , substitute into the formula: , calculate the trigger index , when the trigger index is greater than the index threshold The smart speaker state is defined as triggered when is the baseline length, and is a constant.

[0046] The operation management module includes a voiceprint comparison unit, a command retrieval unit and a state setting unit.

[0047] The voiceprint comparison unit is used to match the associated user. When the smart speaker is in the triggered state, the time The voiceprint characteristics of the subsequent audio Calculate the similarity with the voiceprint features in each database respectively, get the average similarity of each database, mark the database with the highest average similarity, and mark the database when the average similarity is greater than the similarity threshold. When , the smart speaker state is defined as interaction, and the user corresponding to the database is marked as the associated user.

[0048] The command retrieval unit is used to retrieve and execute commands. When the state of the smart speaker is interactive, a term set is established, and the terms in all the command term records corresponding to the associated user are put into the term set, and the time is The audio is then converted into text. , each term and sentence in the term set Calculate the text similarity separately, mark the terms with the highest text similarity, and when the text similarity of the marked terms is greater than the similarity threshold Record time , the smart speaker executes the instructions corresponding to the marked terms.

[0049] The status setting unit is used to set the status of the smart speaker. Real-time update time , get the current time Substituting into the formula: Calculate the state index of the smart speaker at different times ,in, is a constant; set the state index threshold. When the state index is less than the state index, the smart speaker state is defined as triggered; when the state index is not greater than zero, cancel all state definitions of the smart speaker; when the visual area does not contain the position coordinates , unmarks the object.

[0050] The automatic storage module is used to store the voiceprint features and sentence information in the interaction process in the voiceprint feature library and historical records in real time.

[0051] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0052] 1. Activation without wake-up words: In this application, the smart speaker is activated by analyzing the changes in the object's line of sight angle, the changes in the distance from the smart speaker, and the changes in sound intensity. This takes advantage of people's habitual actions when using smart speakers, which is more user-friendly than the wake-up word activation of traditional technologies.

[0053] 2. Privacy and security protection: In this application, the smart speaker does not convert the voice in the audio into text before the interactive state, but uses the method of retrospective audio recording to convert text after entering the interactive state. Compared with the real-time voice-to-text conversion of traditional technology, it has higher privacy and security and can save more computing resources.

[0054] 3. Intelligent interaction state: In this application, when the smart speaker is in the interactive state, the time difference of the last command and the change of the trigger index are used to reset the state. When the object does not issue a command for a long time or actively deviates from the line of sight, the smart speaker state is reset. Compared with the traditional technology, each communication needs to include the setting of a wake-up word, which is more intelligent.

[0055] In summary, compared with traditional technologies, the present invention has the advantages of no wake-up word activation, privacy and security protection, and intelligent interactive state, and can improve the wake-up efficiency of smart speakers. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0057] Figure 1 It is a flow chart of a method for controlling and managing a speaker based on data analysis of the present invention;

[0058] Figure 2 It is a structural schematic diagram of the speaker control management system based on data analysis of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] See also Figure 1 The present invention provides a speaker control management method based on data analysis, the method comprising the following steps:

[0061] S100 collects the voiceprint feature library and historical records stored in the smart speaker, and uses different sensors to collect audio and image information in a specified area in real time, and comprehensively analyzes the audio and image to determine whether an object exists.

[0062] S200. Analyze the sight range of the object through the head posture detection algorithm to establish a visual area, mark the object when there is a smart speaker in the visual area, calculate the trigger index according to the change in sound intensity and determine whether to trigger the smart speaker.

[0063] S300, the smart speaker recognizes the voiceprint features in the audio, obtains the corresponding historical records, executes the corresponding instructions according to the historical records and enters the interactive state, calculates the state index through the instruction time difference and sets the smart speaker state.

[0064] S400: storing the voiceprint features and sentence information during the interaction process into a voiceprint feature library and a historical record respectively.

[0065] In S100, the voiceprint feature library refers to a database that stores voiceprint features. Each user corresponds to a database that stores different voiceprint features. The historical record refers to the user's command phrase record. Each user contains different command phrase records, and each command phrase record includes commands and phrases. The designated area refers to the area where the smart speaker is used. The camera collects image information in the designated area. The image is a real-time video, and the microphone collects audio information in the designated area. The steps to determine whether an object exists are as follows:

[0066] S101, the microphone collects the audio in the specified area in real time and calculates the sound intensity, and records the time when the sound intensity is greater than the intensity threshold , automatically controls the camera to start recording video in the specified area.

[0067] S102, using OpenCV technology to decompose the video into single-frame images, screening all single-frame images, using edge detection method to remove blurred single-frame images; using YOLOv5 human body detection algorithm to perform target recognition on the remaining single-frame images, and taking the recognized human targets as objects. If no object is recognized in the specified area, the camera is controlled to be turned off.

[0068] A command is an operation instruction that can be directly recognized and executed by a smart speaker. A term is a textual expression made by a user when issuing a command, which is obtained by converting the voice in the collected audio information into text. Each command corresponds to several terms, which can represent the same command, and different terms have different sentence structures or adjectives.

[0069] In S200, the specific steps are as follows:

[0070] S201, using a key point detection algorithm to identify different parts of the object and mark the key points, and obtaining the head deflection angle by analyzing the changes in the positions of these key points, and then combining the head posture detection algorithm to analyze and obtain the object's line of sight direction.

[0071] Each part of the object has a different number of key points, and key points are usually set at different positions of the main parts of the human body. The head deflection angle and line of sight direction are analyzed based on the position or angle changes of these key points.

[0072] For example, by setting key points on the shoulders and connecting the two shoulder key points, the direction of the subject's shoulders can be identified by the change in angle, and the movement of the subject can be identified by the change in position. Alternatively, by setting key points on the back of the head, face, and ears in the head area, the direction of the subject's face can be analyzed by the presence of key points on the back of the head or face, the deflection angle of the subject's head can be identified by the change in angle after connecting the two ear key points, and the forward or backward tilt of the subject's head can be identified by the change in position.

[0073] S202: Obtaining the position coordinates of the object , set the horizontal viewing angle R and sensing distance L, with the position coordinates With the sensor distance L as the center and the radius as the radius, a sector area with an angle of R is divided in the specified area in the direction of sight as the object's visual area. A line segment with a distance of L is intercepted from the object's position along the direction of sight as the visual line. The visual line is the middle line of the visual area, and the visual area changes dynamically as the object moves or the head deflects.

[0074] The horizontal viewing angle and sensing distance are set in advance by the administrator. The specific values ​​are based on the attention angle of the human eye in the horizontal dimension and the anti-miscontrol distance of the smart speaker.

[0075] The attention angle of the human eye in the horizontal dimension refers to the range that the human visual system can focus and pay attention to in the horizontal direction. This angle is usually used to describe the range that the human eye can cover in the horizontal direction, that is, the area that the human eye can observe and pay attention to through eye movement and adjustment without turning the head.

[0076] The anti-miscontrol distance of smart speakers refers to the effective range within which the device can recognize and respond to human commands when using smart speakers. This distance usually refers to the distance between a person and the smart speaker. Beyond this distance, the smart speaker may not be able to accurately recognize user commands or respond effectively.

[0077] S203: Obtain the location coordinates of the smart speaker in the specified area , real-time analysis of the dynamic changes in the visual area of ​​the object, the visual area contains the position coordinates Then mark the object and record the time ,according to arrive The average sound intensity is calculated from the audio information during this period .

[0078] The visual area of ​​the marker object always contains the position coordinates , when the visual area no longer contains the position coordinates , the object is unmarked.

[0079] S204: Use the line segment connecting the position of the marked object and the position of the smart speaker as the baseline, and obtain the angle between the visual line and the baseline , and time The sound intensity after , set the sound intensity increase value , calculate the trigger index of the marked object at different times in real time, when the trigger index is greater than the index threshold The smart speaker state is defined as triggered when, the formula is as follows:

[0080] ;

[0081] In the formula, is the trigger index, is the baseline length, and is a constant.

[0082] When the baseline length is smaller, the angle The smaller the sound or the higher the sound intensity, the greater the trigger index. The trigger index is used to quantify the likelihood that the subject is ready to use the smart speaker.

[0083] In S300, the specific steps are as follows:

[0084] S301: When the state of the smart speaker is triggered, the voiceprint recognition algorithm is used to extract the time The voiceprint characteristics of the subsequent audio , the voiceprint feature Match all the voiceprint features in the voiceprint feature library and calculate the similarity. Sum the similarities of all voiceprint features in the same database and divide it by the total number of voiceprint features in the database to get the average similarity. Mark the database with the highest average similarity. When the average similarity of the marked database is greater than the similarity threshold When , the smart speaker state is defined as interaction, and the user corresponding to the database is marked as the associated user.

[0085] Voiceprint feature matching uses text-independent voiceprint recognition, which refers to the use of individual voice features for identity verification or identification without relying on specific text content. Voiceprint recognition is a biometric technology that identifies people by analyzing the tone, timbre or speaking speed of voice features.

[0086] S302: When the state of the smart speaker is interactive, a term set is established, and the terms in all the command term records corresponding to the associated user in the historical records are put into the term set, and the time is The audio is then converted into text. , each term and sentence in the term set Calculate the text similarity separately, mark the terms with the highest text similarity, and when the text similarity of the marked terms is greater than the similarity threshold Record time , the smart speaker executes the instructions corresponding to the marked terms.

[0087] S303, continuously analyzing the speech in the audio information, substituting it into step S302 to calculate the text similarity and update the time ; Get the current time Substitute it into the formula to calculate the state index of the smart speaker at different times:

[0088] ;

[0089] In the formula, is the state index, is a constant. Set the state index threshold. When the state index is less than the state index, the smart speaker state is defined as triggered. When the state index is not greater than zero, all state definitions of the smart speaker are cancelled. When the visual area does not contain the position coordinates , unmarks the object.

[0090] The state of the smart speaker is set by the state index. In the interactive state, the smart speaker no longer performs voiceprint recognition, but only performs voice-to-text conversion, recognizes and executes commands. In the triggered state, the smart speaker no longer performs voice-to-text conversion, recognizes and executes commands, but only performs voiceprint recognition to determine whether to switch states. In the none state, only the trigger index of the marked object is calculated to determine whether to set the smart speaker to the triggered state. The visual area does not contain position coordinates Automatically unmark objects when

[0091] In S400, the voiceprint features during the interaction between the marked object and the smart speaker are stored in the database of the associated user in the voiceprint feature library, and the sentence information and the corresponding instructions are recorded as the instruction terms of the associated user and stored in the historical record.

[0092] See also Figure 2 The present invention provides a speaker control management system based on data analysis, which includes a data acquisition module, an environment analysis module, an operation management module and an automatic storage module.

[0093] The data acquisition module is used to collect voiceprint feature libraries and historical records, as well as audio and image information in the specified area. The environmental analysis module determines whether there is an object through audio and image, establishes a visual area for the object, and calculates the trigger index based on the change in sound intensity when there is a smart speaker in the visual area and determines whether to trigger the smart speaker. The operation management module performs voiceprint recognition on the audio, obtains the historical records corresponding to the voiceprint features, executes instructions according to the historical records and enters the interactive state, calculates the state index through the time difference of the instructions, and thus sets the state of the smart speaker. The automatic storage module is used to store information in the interactive process in real time.

[0094] The data collection module includes a feature library collection unit, a history record collection unit, an image information collection unit and an audio information collection unit.

[0095] The feature library acquisition unit is used to collect and store the database of voiceprint features. Each user corresponds to a database, and different voiceprint features are stored in the database. The history record acquisition unit is used to collect the command terminology records of each user. Each command terminology record includes the command and terminology. The image information acquisition unit collects the image information in the specified area through the camera, and the image is real-time video. The audio information acquisition unit collects the audio information in the specified area through the microphone.

[0096] The environment analysis module includes an object analysis unit and a wake-up judgment unit.

[0097] The object analysis unit is used to identify objects in a specified area. The sound intensity is calculated through audio information. If the sound intensity is greater than the intensity threshold, the time is recorded. The camera is controlled to record the video. The video is deframed into single-frame images, and after the clear single-frame images are screened out, the human body detection algorithm is used for target recognition, and the recognized human body target is regarded as the object.

[0098] The wake-up judgment unit is used to judge whether to wake up the smart speaker.

[0099] First, the key point detection algorithm is used to mark the key points of different parts of the object, and the line of sight of the object is obtained based on the changes in the positions of these key points.

[0100] Secondly, set the horizontal viewing angle R and sensing distance L to the object's position coordinates. As the center of the circle, L is the radius, and a sector area with an angle of R is divided in the specified area towards the line of sight as the visual area of ​​the object, and the middle line of the visual area is the visual line; when the visual area contains the position coordinates of the smart speaker When the object is marked and the time is recorded ,calculate arrive The average sound intensity during this period .

[0101] Finally, the line segment connecting the position of the marked object and the position of the smart speaker is used as the baseline to obtain the angle between the visual line and the baseline. , and time The sound intensity after , set the sound intensity increase value , substitute into the formula: , calculate the trigger index , when the trigger index is greater than the index threshold The smart speaker state is defined as triggered when is the baseline length, and is a constant.

[0102] The operation management module includes a voiceprint comparison unit, a command retrieval unit and a state setting unit.

[0103] The voiceprint comparison unit is used to match the associated user. When the smart speaker is in the triggered state, the time The voiceprint characteristics of the subsequent audio Calculate the similarity with the voiceprint features in each database respectively, get the average similarity of each database, mark the database with the highest average similarity, and mark the database when the average similarity is greater than the similarity threshold. When , the smart speaker state is defined as interaction, and the user corresponding to the database is marked as the associated user.

[0104] The command retrieval unit is used to retrieve and execute commands. When the state of the smart speaker is interactive, a term set is established, and the terms in all the command term records corresponding to the associated user are put into the term set, and the time is The audio is then converted into text. , each term and sentence in the term set Calculate the text similarity separately, mark the terms with the highest text similarity, and when the text similarity of the marked terms is greater than the similarity threshold Record time , the smart speaker executes the instructions corresponding to the marked terms.

[0105] The status setting unit is used to set the status of the smart speaker. Real-time update time , get the current time Substituting into the formula: Calculate the state index of the smart speaker at different times ,in, is a constant; set the state index threshold. When the state index is less than the state index, the smart speaker state is defined as triggered; when the state index is not greater than zero, cancel all state definitions of the smart speaker; when the visual area does not contain the position coordinates , unmarks the object.

[0106] The automatic storage module is used to store the voiceprint features and sentence information in the interaction process in the voiceprint feature library and historical records in real time.

[0107] Example 1: Assume that there is an object in the specified area, the visual area of ​​the object includes the location of the smart speaker, the average sound intensity is 50db, the sound intensity increase value is 30db, the current sound intensity is 60db, the horizontal viewing angle is 120°, the angle between the visual line and the baseline is 15°, the sensing distance is 5m, and the baseline length is 3m. is 1, is 0.1, substitute it into the formula to calculate the trigger index:

[0108] .833;

[0109] When the index threshold is 0.3, the smart speaker state is defined as triggered.

[0110] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0111] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A speaker control management method based on data analysis, characterized in that: The method comprises the following steps: S100, collect the voiceprint feature library and historical records stored in the smart speaker, and use different sensors to collect audio information and image information in a specified area in real time, and comprehensively analyze the audio and image to determine whether there is an object; S200, analyzing the sight range of the object through the head posture detection algorithm to establish a visual area, marking the object when there is a smart speaker in the visual area, calculating the trigger index according to the change of sound intensity and determining whether to trigger the smart speaker; S300, the smart speaker recognizes the voiceprint features in the audio, obtains the corresponding historical records, executes the corresponding instructions according to the historical records and enters the interactive state, calculates the state index through the instruction time difference and sets the smart speaker state; S400, storing the voiceprint features and sentence information during the interaction process into a voiceprint feature library and a historical record respectively; In S200, the specific steps are as follows: S201, using a key point detection algorithm to identify different parts of the object and mark the key points, analyzing the changes in the positions of these key points to obtain the head deflection angle, and then combining the head posture detection algorithm to analyze and obtain the line of sight direction of the object; S202: Obtaining the position coordinates of the object , set the horizontal viewing angle R and sensing distance L, with the position coordinates With the center as the circle and the sensing distance L as the radius, a sector area with an angle of R is divided in the specified area towards the line of sight as the object's visual area; a line segment with a distance of L is intercepted from the object's position along the line of sight as the visual line; the visual line is the middle line of the visual area, and the visual area changes dynamically as the object moves or the head deflects; S203: Obtain the location coordinates of the smart speaker in the specified area , real-time analysis of the dynamic changes in the visual area of ​​the object, the visual area contains the position coordinates Then mark the object and record the time ,according to arrive The average sound intensity is calculated from the audio information during this period ; S204: Use the line segment connecting the position of the marked object and the position of the smart speaker as the baseline, and obtain the angle between the visual line and the baseline , and time The sound intensity after , set the sound intensity increase value , calculate the trigger index of the marked object at different times in real time, when the trigger index is greater than the index threshold The smart speaker state is defined as triggered when, the formula is as follows: ； In the formula, is the trigger index, is the baseline length, and is a constant.

2. The method for controlling and managing a speaker based on data analysis according to claim 1, characterized in that: In S100, the voiceprint feature library refers to a database storing voiceprint features, each user corresponds to a database, and different voiceprint features are stored in the database; the historical record refers to the user's command term record, each user contains different command term records, and each command term record includes commands and terms; the designated area refers to the use area of ​​the smart speaker, and the image information in the designated area is collected by the camera, and the image is a real-time video, and the audio information in the designated area is collected by the microphone; the steps for determining whether an object exists are as follows: S101, the microphone collects the audio in the specified area in real time and calculates the sound intensity, and records the time when the sound intensity is greater than the intensity threshold , automatically controls the camera to start recording video in the specified area; S102. Use OpenCV technology to decompose the video into single-frame images, screen all the single-frame images, and use edge detection method to remove blurred single-frame images; use YOLOv5 human detection algorithm to perform target recognition on the remaining single-frame images, and use the recognized human targets as objects.

3. The method for controlling and managing a speaker based on data analysis according to claim 2, characterized in that: In S300, the specific steps are as follows: S301: When the state of the smart speaker is triggered, the voiceprint recognition algorithm is used to extract the time The voiceprint characteristics of the subsequent audio , the voiceprint feature Match all the voiceprint features in the voiceprint feature library and calculate the similarity. Sum the similarities of all voiceprint features in the same database and divide it by the total number of voiceprint features in the database to get the average similarity. Mark the database with the highest average similarity. When the average similarity of the marked database is greater than the similarity threshold When , the smart speaker state is defined as interaction, and the user corresponding to the database is marked as the associated user; S302: When the state of the smart speaker is interactive, a term set is established, and the terms in all the command term records corresponding to the associated user in the historical records are put into the term set, and the time is The audio is then converted into text. , each term and sentence in the term set Calculate the text similarity separately, mark the terms with the highest text similarity, and when the text similarity of the marked terms is greater than the similarity threshold Record time , the smart speaker executes the instruction corresponding to the marked term; S303, continuously analyzing the speech in the audio information, substituting it into step S302 to calculate the text similarity and update the time ; Get the current time Substitute it into the formula to calculate the state index of the smart speaker at different times: ； In the formula, is the state index, is a constant; set the state index threshold. When the state index is less than the state index, the smart speaker state is defined as triggered; when the state index is not greater than zero, cancel all state definitions of the smart speaker; when the visual area does not contain the position coordinates , unmarks the object.

4. The method for controlling and managing a speaker based on data analysis according to claim 3, characterized in that: In S400, the voiceprint features during the interaction between the marked object and the smart speaker are stored in the database of the associated user in the voiceprint feature library, and the sentence information and the corresponding instructions are recorded as the instruction terms of the associated user and stored in the historical record.

5. The speaker control management system based on data analysis is characterized by: The system includes a data acquisition module, an environmental analysis module, an operation management module and an automatic storage module; The data collection module is used to collect voiceprint feature libraries and historical records, as well as audio and image information within a specified area; The environment analysis module determines whether there is an object through audio and images, establishes a visual area for the object, and calculates the trigger index according to the change in sound intensity when there is a smart speaker in the visual area, and determines whether to trigger the smart speaker; the operation management module performs voiceprint recognition on the audio, obtains the historical records corresponding to the voiceprint features, executes instructions according to the historical records and enters the interactive state, calculates the state index through the time difference of the instructions, and thus sets the state of the smart speaker; the automatic storage module is used to store information in the interactive process in real time; The environment analysis module includes an object analysis unit and a wake-up judgment unit; The object analysis unit is used to identify objects in a specified area; the sound intensity is calculated through audio information, and the time is recorded if the sound intensity is greater than the intensity threshold. And control the camera to record video; decode the video into single-frame images, filter out clear single-frame images, and use human detection algorithm to identify the target, and the identified human target is regarded as the object; The wake-up judgment unit is used to judge whether to wake up the smart speaker; First, the key points of different parts of the object are marked using a key point detection algorithm, and the object's line of sight is obtained based on the change in the position of these key points. Secondly, set the horizontal viewing angle R and sensing distance L to the object's position coordinates. As the center of the circle, L is the radius, and a sector area with an angle of R is divided in the specified area towards the line of sight as the visual area of ​​the object, and the middle line of the visual area is the visual line; when the visual area contains the position coordinates of the smart speaker When the object is marked and the time is recorded ,calculate arrive The average sound intensity during this period ; Finally, the line segment connecting the position of the marked object and the position of the smart speaker is used as the baseline to obtain the angle between the visual line and the baseline. , and time The sound intensity after , set the sound intensity increase value , substitute into the formula: , calculate the trigger index , when the trigger index is greater than the index threshold The smart speaker state is defined as triggered when is the baseline length, and is a constant.

6. The speaker control and management system based on data analysis according to claim 5 is characterized in that: The data collection module includes a feature library collection unit, a history record collection unit, an image information collection unit and an audio information collection unit; The feature library collection unit is used to collect and store the database of voiceprint features. Each user corresponds to a database, and different voiceprint features are stored in the database; the historical record collection unit is used to collect the command terminology records of each user, and each command terminology record includes instructions and terms; the image information collection unit collects image information in the specified area through a camera, and the image is real-time video; the audio information collection unit collects audio information in the specified area through a microphone.

7. The speaker control and management system based on data analysis according to claim 6 is characterized in that: The operation management module includes a voiceprint comparison unit, a command retrieval unit and a state setting unit; The voiceprint comparison unit is used to match the associated user; when the state of the smart speaker is triggered, the time The voiceprint characteristics of the subsequent audio Calculate the similarity with the voiceprint features in each database respectively, get the average similarity of each database, mark the database with the highest average similarity, and mark the database when the average similarity is greater than the similarity threshold. When , the smart speaker state is defined as interaction, and the user corresponding to the database is marked as the associated user; The command retrieval unit is used to retrieve and execute commands; when the state of the smart speaker is interactive, a term set is established, and the terms in all the command term records corresponding to the associated user are put into the term set, and the time is The audio is then converted into text. , each term and sentence in the term set Calculate the text similarity separately, mark the terms with the highest text similarity, and when the text similarity of the marked terms is greater than the similarity threshold Record time , the smart speaker executes the instruction corresponding to the marked term; The status setting unit is used to set the status of the smart speaker; real-time update time , get the current time Substituting into the formula: Calculate the state index of the smart speaker at different times ,in, is a constant; set the state index threshold. When the state index is less than the state index, the smart speaker state is defined as triggered; when the state index is not greater than zero, cancel all state definitions of the smart speaker; when the visual area does not contain the position coordinates , unmarks the object.

8. The speaker control and management system based on data analysis according to claim 7 is characterized in that: The automatic storage module is used to store the voiceprint features and sentence information in the interaction process in the voiceprint feature library and historical records in real time.

Images