Systems and methods and software programs for improving device performance

Abstract

The present invention relates to a system and corresponding method, as well as computer-implemented software, for improving the performance of at least one electronic device. The device includes at least one sensor and a model that defines the reaction of the device in response to data generated by the at least one sensor. The system includes a model generator configured to analyze the data generated by the sensor and the reaction to the data, and to register an error in the reaction compared to an intended reaction. The model generator is configured to adjust the model by minimizing an error between an actual output of the model of the device and a desired output of the model based on a set of samples recorded on the at least one electronic device. The recorded samples include information about which of a number of predefined states the device was in when an error occurred, a method regarding the time when the error occurred, and information about the number of errors at a specified time and / or state.

Description

[Technical Field]

[0001] The present invention relates to a system and method for analyzing the performance of a device. [Background technology]

[0002] Advanced electronic devices are often built for specific uses. In this case, the device is adapted to respond to real-world conditions. This may include devices where certain assumptions are made to enhance relevant signals and suppress other signals, such as background noise, such as in mobile phones or voice recognition devices, or may include automatic functions that can accommodate transitions between different modes of use. While these parameters are chosen to be as realistic as possible, background noise and voice commands may differ from the real-world conditions assumed. While many electronic devices have systems for updating their software, it can be difficult to obtain sufficient good information about the device's performance, leading to inadequate updates.

[0003] Variations in the acoustic configuration of devices require that each device be manually adjusted and calibrated to ensure consistent performance. Naturally, this is a time-consuming and costly process. It is therefore an object of the present invention to simplify this process. Summary of the Invention [Problem to be solved by the invention]

[0004] It is an object of the present invention to provide a system for improving the performance of electronic devices. This object is achieved according to the appended claims. [Means for solving the problem]

[0005] According to the present invention, a device can transmit device-specific information to a preferably cloud-based platform, which is provided with specific product specifications. Based on this information, the system can generate a machine learning model using machine learning techniques. The platform receives sampled performance information from the device, compares it with specifications provided by the manufacturer, and adjusts the model, which is fed back to the device as an update. In this way, the performance of the device can be adjusted in the platform using machine learning procedures.

[0006] For example, device-specific software may report the measured ultrasonic activity in the device's environment. Specifications provided by the device's manufacturer are based on expected activity. This means that the device may not be adequately adapted to the actual environment. For example, a device may be adapted to handle a small number of devices using the same ultrasonic measurements in the vicinity, but in some situations, such as a concert, the background sound profile may be significantly different, and interference may reduce the device's performance.

[0007] Another example may relate to a device used for playback in a smart speaker system, where the system needs to automatically switch to a different speaker when moving to a different area. Reported errors may be used to adjust the characteristics of the software appliance to improve the transition.

[0008] Yet another example relates to the change in performance and functionality of the screen, microphone and / or speaker when the mobile phone is brought closer to the user's ear.

[0009] The device may report measurements representative of that moment, and the analysis may conclude that the device design needs to be changed and report this to the manufacturer.

[0010] Preferably, the analysis is performed on a cloud-based system that is in communication with the devices, receiving information from the manufacturer and reporting information to the manufacturer.

[0011] The invention may also be used in devices such as mobile phones that are controlled by user gestures, reporting measured deviations between pre-programmed gestures interpreted by the device and the actual movements of one or more particular users, allowing the system to adapt to them. [Brief explanation of the drawings]

[0012] The invention will now be explained in more detail by way of example with reference to the accompanying drawings, in which: [Figure 1] 1 is a diagram showing a schematic diagram of the present invention; [Figure 2] 1 illustrates a preferred embodiment of the present invention in more detail. [Figure 3] FIG. 1 illustrates process steps in a machine learning sequence according to the present invention. [Figure 4] FIG. 10 illustrates the definition of logged errors used by the system. DETAILED DESCRIPTION OF THE INVENTION

[0013] As shown in Figure 1, device manufacturers and application developers can take advantage of presence and gesture sensing capabilities without requiring expertise in acoustics or signal processing. This may be performed via a cloud-based machine learning platform, which uses advanced machine learning techniques to detect user presence, estimate room occupancy, and provide gesture recognition.

[0014] FIG. 1 illustrates the architecture of the technology for a device manufacturer 1 and a device 3 communicating with a preferably cloud-based machine learning service platform 2. The device manufacturer 1 simply defines the operating environment of their product (e.g., memory, storage, computing power, power consumption, mechanical setup, and component characteristics) and selects the desired performance in terms of software range and sensitivity. The invention then provides a device-specific software integration package and a set of machine learning classifiers provided by the system. Each classifier is a set of parameters used to map reflected ultrasound signals into a set of categories (e.g., one nearby user, two nearby users, or no user). These classifiers require little processing power and can therefore run on any smart device.

[0015] Naturally, machine learning classifiers require large amounts of device-specific data for maximum accuracy. Training is performed on Service Platform 2, which hosts the data used to train the classifiers and provides updates based on new information uploaded to the machine learning training center. The cloud also contains advanced signal processing libraries (which preprocess the acoustic data, reducing the amount of memory and processing power required for classification) and empirical data on hardware performance (i.e., a database of ultrasound recordings showing the performance characteristics of various acoustic components and the specific effects of mechanical design). These libraries work together to optimize overall system performance.

[0016] Once a system according to the present invention is installed on or configured to communicate with a smart device, its range, field of view, and performance can be further adjusted (by the manufacturer or end user) via an application programming interface (API). The device may also be able to correlate results with other inputs (such as tactical interactions) to determine the accuracy of presence determination and optimize itself. This essentially customizes the user's experience to their environment and improves future responses.

[0017] The invention provides a cloud-based platform that includes machine learning algorithms, recording databases, and signal processing tools that work together to reduce the deployment time required for standard audio and acoustic configurations, making them more "plug and play."

[0018] A more detailed system overview is shown in Figure 2.

[0019] Manufacturer 1 first provides the system with instructions on how the device should react to specific sensor outputs from sensors included in device 3. The manufacturer also provides the machine learning system with the necessary computer code, sensor characteristics, model adjustment limits, and a communication protocol for the machine learning system to communicate with the device. The manufacturer may also provide predetermined limits for model adjustment and direct communication with the device to receive the same data from the device that is communicated to the machine learning system for, e.g., evaluation or development purposes.

[0020] Manufacturer 1 also provides the necessary information to Machine Learning Platform 2, such as the model, its tunability within the model, and its expected response to sensor outputs within the device, including the code required to communicate with each device.

[0021] The manufacturer 1 may also be able to receive updated models from the machine learning service model generator 2b, allowing it to evaluate the models in relation to the data received from the device 3.

[0022] The device 3 may comprise two different units 3a and 3b, including a software application 3a adapted to collect data from the device and a processor / engine 3b. The recorded data may be from sensors or other performance measurements obtained, for example, by comparing different data types registered in the same situation, or user feedback received via an appropriate user interface regarding the measurements or situation. These data are reported to the cloud-based storage facility 2a for further processing and evaluation. In addition to the data received from the device 3, the storage facility 2a may also include data collected from other similar devices under relevant conditions.

[0023] In addition to data measured from use of the device, the engine or processor 3b within the device may report its performance to the cloud-based machine learning platform 2.

[0024] Engine 3b contains code that allows the integration of new models and parameters provided by the system, with the aim of improving the device performance in a particular operation through a system learning process.

[0025] The cloud-based system 2 is adapted to use the information stored in the storage facility 2a and information about the models in the devices to build, in the model generator 2b, an updated model from the characteristics and performance of the devices or responses from the devices, as well as information possibly obtained from similar devices received from manufacturers. The resulting model may then be provided to the device engine 3b, for example as a software update, to adjust the performance of the device 3.

[0026] The model may also be simulated and reported to the manufacturer 1 via a visual user interface for further evaluation.

[0027] The system according to the invention may also provide testing means for the manufacturer 1 to send test cases to the device in order to check the performance of the device, for example to detect errors in the software 3a or hardware 3b.

[0028] As mentioned above, the cloud-based system may be adapted to provide support services for different types of equipment based on engine codes and control parameters from the equipment manufacturer's database and data storage, including data collected from the equipment 3 and measurement data and performance provided by the equipment, where both the manufacturer and the equipment are provided with an interface for an appropriate communication interface to communicate with the system.

[0029] Monitoring such data can provide the basis for machine learning, where the system can model and update the performance of the device based on new data received from the device and manufacturer. This may include adjusting control parameters in the model relative to initial parameters provided by the manufacturer. This allows the performance of the device to be changed by implementing the new model and parameters in the device engine or processor that controls its operation. As discussed above, this provides a streamlined and cost-effective way to improve the device.

[0030] An example of the process is shown below.

[0031] An exemplary process according to the present invention implemented in a cloud-based system 2 is shown in Figure 3, where the process includes: (10) collecting data from the device software 3a of any number of sensors in the device 3 to cloud storage, the type of sensor being selected from, for example, a camera, an acoustic sensor, an inertial sensor, a radar, or an optical measurement; (11) tagging the cloud data storage 2a with details of the tagged data, such as data type and time; (12) scrubbing the stored data using a manufacturer-defined scrubbing routine; (13) preprocessing the sampled data for modeling; (14) constructing a model representing the data based on the preprocessed data; (15) selecting an optimal model based on device performance through an iterative process; (16) introducing the obtained new model into the device; Includes:

[0032] Selecting a model in step 15 may be performed by software rules such as analyzing user statistics, registering user feedback, or comparing the measured response to expected performance under known circumstances, and may also include measuring or selecting additional data, for example from other sensors.

[0033] An example of how the data is used is shown in Figure 4.

[0034] Figure 4 shows the transition between two states 21a and 21b. These are different situations where the device should react differently, such as when a mobile phone user lifts the phone off a table versus holding it close to their ear. In this case, the mobile phone can react by activating the speaker and microphone or by deactivating the touch sensitivity of the screen. This can be achieved using the mobile phone's motion and orientation sensors.

[0035] In state 21a, the sensor can sense the steady-state condition 22a, where the mobile phone is resting on the table, with a high degree of certainty and a low probability of error in reporting the mobile phone's state. Similarly, active use of the mobile phone, such as in state 21b, such as holding the mobile phone close to the ear, results in a steady-state period 22e with a low number of false reports. However, during the process of bringing the mobile phone closer to the ear, there is a transitional state period 22c where the sensor has difficulty reporting the correct state, and where it is difficult to precisely adjust the correct moment or location that defines the transition. This transitional state period may have a known or predetermined length.

[0036] Also, according to the present invention, the transition from the initial state 21a to the next state 21b includes a pre-transition state period 22b and a post-transition state period 22d. The pre-transition state and the post-transition state may be defined according to the distance to the expected transition state period 22c or the time before and after the occurrence of the transition state period 22c. Needless to say, the specific time and distance will vary depending on the type of transition. Also, the time and distance before and after the transition may be different before and after the transition state period 22c. Hereinafter, the periods 22a to 22e will also be referred to as "slices."

[0037] Referring to the example described above, a transition may be defined as the moment a mobile phone is lifted from a table toward the ear and the mobile phone assumes a substantially vertical position. The pre-transition time represents the first half of the expected time during which the mobile phone measures movement, and the post-transition time represents the second half. During the pre-transition and post-transition periods, the number of errors should be small, but the greater the distance, e.g., in time (or location), to the expected transition, the greater the likelihood of an error being real.

[0038] Once the steady state, pre-transition state, post-transition state, and transition state are established, operational errors may be detected, either manually by a user or by analysis of the signals, and predetermined error values ​​may be determined. The error values ​​may be of three different types, subdivided based on size and / or duration, as follows: [Error Count (Boolean)] Error Boolean (errors detected regardless of size / duration of the error) Error Boolean (errors smaller than a given size or duration) Error Boolean (errors larger than a given size or longer than a given duration) [Error Count (Total)] Error count (errors detected regardless of size / duration of the error) Error count (errors smaller than a certain size or duration) Error counts (errors larger than a certain size or longer than a certain duration) Also, the type of error can be found based on the state in which the error occurred. These detected errors may then be used in machine learning.

[0039] In addition to the type of error, the presence or location of the error within slices 22a-22e may be used to obtain information about the error experienced, such as: Any time within a slice Whole slice the beginning of the slice End of slice During a slice, regardless of the first and last error Other combinations may also be defined, such as an error occurring at the beginning of a slice but not occurring in the middle or at the end.

[0040] Based on these detected errors it is possible to define different situations, for example: [Example 1] Technical definition: Slice: 22 days after transition from status 0 (no detection) to status 1 (detection) Error value: Boolean Error type: Error in the entire slice interpretation: The system did not detect the transition from Status 0 to Status 1 in enough time. this is: · It may indicate a failure to detect proximity when holding the phone to your ear. If the region contains multiple defined detection regions, it may indicate that the entry into a new region could not be detected when the user enters a new region, for example when playing from a device using a smart speaker system. [Example 2] Technical definition: Slice: Before transition from status 0 (not detected) to status 1 (detected) 22b Error value: Duration Error type: There is an error at the end of the slice. interpretation: The system detected the transition from Status 0 to Status 1 a little early, by how much, as indicated by the duration. this is: · Shows the duration of early detection when holding the phone close to your ear. [Example 3] Technical definition: Slice: Steady state 22a Error value: Count Error type: in the middle of a slice, independent of the first and last errors interpretation: Except for late detection (counting begins after reaching status 1) and early transitions (counting ignores transitions to another state if they occur at the end of a slice), the system has lost status 1 at least once. The error count provides the number. this is: · The phone shows several loss of functionality, including flickering during calls. · When using playback from a smart speaker device, it shows that playback stays within a region and jumps to another region.

[0041] Based on the registration data on the number of error messages and occurrence of errors stored in the cloud-based storage system 2a, and the model 2b with related parameters in the cloud-based system provided from the manufacturer's database, the machine-based learning is adapted to propose and compare models in a simulation and evaluation process to find the model and related parameters that can most reduce the number of errors. In this way, for example, if a large number of errors are registered due to slow mode switching of the mobile phone when held close to the ear, the model can be adjusted to switch more quickly if the corresponding action is registered.

[0042] The selected model is then sent to the manufacturer for evaluation, thus providing the manufacturer with a new model that is adjusted according to the actual use of the device, resulting in improved performance under changing conditions.

[0043] Based on this model, the manufacturer can use it to test the hardware or software sent to the device's software application 3a and / or hardware 3b, or reprogram the device with hardware or firmware according to the new model.

[0044] It may therefore be concluded that one aspect of the present invention provides a computer system and corresponding method for optimizing device performance, the system including a data storage adapted to receive predetermined information regarding the device's processor performance and data collected from sensors within the device, such as acoustic detection devices and motion sensors, indicative of the device's use and performance under realistic conditions.

[0045] More specifically, the present invention relates to a computer system for improving the performance of at least one electronic device. The device includes sensors and software that samples predetermined information about selected operations performed by the device and a predetermined model that specifies the device's response in predetermined situations. The sensors may include measuring instruments such as cameras, acoustic sensors, inertial sensors, radar, or optical measurements. The model is initially provided by the manufacturer, which specifies the device's planned performance under certain conditions.

[0046] The computer system is also configured to receive and store sampled information from the device and device information (including model) about the device from the manufacturer.

[0047] The information sampled by the device also includes reported errors and / or deviations from model performance related to the sampled information. Errors may be detected by the device software or reported by a user via an appropriate interface.

[0048] The computer system is adapted to generate an adjusted model based on the reported errors and sensor measurements and the previous model, and to generate an updated model based on deviations between the initial model and the measured performance. The updated model may be sent to the device and reported to the manufacturer, who can further reprogram and / or test the performance of the model before being updated and tested on the device via a firmware or software, e.g., application, update.

[0049] Preferably, the sampled information is tagged with data type, time, and / or location tags to ensure a relationship between the sampled and reported information.

[0050] The system may also use a predetermined set of rules to analyze error detections and associated measurements in the sampled information, where the system is configured to iteratively reconfigure the model to reduce the number of error detections.

[0051] The predetermined rule set may include classification of errors according to predetermined steady and transition states, and pre- and post-transition states, where the iterations are configured to reduce errors in the steady, pre-, and post-transition states.

[0052] Preferably, the computer system is a cloud-based computer system, the system comprising: a data storage for receiving measured data from the device and information on performance characteristics of the device, a model generation unit for generating an adjusted model based on an initial model provided by the manufacturer and the data stored in the data storage, and a model evaluation unit for evaluating the generated model and sending the model to the manufacturer.

[0053] The evaluation unit and model generator iteratively regenerates and evaluates the model to generate an improved model.

[0054] The method and computer implemented software according to the present invention comprises: sampling, at at least one device, information relating to error detection and measurements related to the error detection, the detection including errors reported by a user or detected by device software; receiving information sampled by devices in a computer network, the network including storage means containing information relating to device software and performance, and models specifying device performance; adjusting a model at the network based on the reported errors and associated measurements; transmitting the adjusted model to a device, where the device updates its software to correspond to the adjusted model; Includes:

[0055] According to another aspect, the present invention relates to a system for improving the performance of at least one electronic device, where the device includes at least one sensor and a model that defines a response of the device in response to data generated by the at least one sensor.

[0056] The system includes a model generator configured to analyze data generated by the sensor and a response to the data, where the response may be based on an analysis of the device's performance under measured conditions or may be provided via a user interface, and an error in the response compared to an intended response is registered according to the model.

[0057] The model generator is configured to adjust the model by minimizing an error between an actual output of the device model and a desired output of the model based on a set of samples recorded on the at least one electronic device.

[0058] The recorded samples were Information about which of a number of predetermined states the device was in when the error occurred; Information about the time the error occurred, and Information about the number of errors at a specified time and / or state Includes:

[0059] Preferably, the predetermined states relate to transition situations and include a steady state, a pre-transition state, a transition state, and a post-transition state.

[0060] Preferably, the specified time includes the beginning of the state, the middle of the state, the end of the state, the entire state, or a combination thereof, which may define, for example, the duration of a pre-transition state and / or the time within the state to which the reaction refers.

[0061] Thus, the count of errors may include a Boolean count and a summary count, or a duration ratio.

[0062] Preferably, the model generator is a network system in communication with a number of devices, wherein the model generator may be adapted to communicate with a manufacturer that provides an initial model and expected performance of the device, and the device may be provided with a user interface configured to receive error reports from device users.

[0063] The error may be registered in the measured deviation between the expected response and the measured response of the device, or may be reported via a user interface configured to receive a response from a user, where the user reports, for example, a deviation from an intended operating model.

[0064] The system may be configured to analyze error detections and associated measurements in the sampled data based on a predetermined set of rules, where the system is configured to iteratively reconfigure the model to reduce the number of error detections based on a set of compiled responses or continuously based on a set consisting of the most recently received number of responses.

[0065] The sensors in the device may include at least one of a camera, an acoustic sensor, an inertial sensor, a radar, or an optical measurement, and an analysis of the performance of the device may be based on one or more of the sensors during state changes.

[0066] Yet another aspect of the present invention relates to a method for improving the performance of a particular type of device, the method comprising: sampling, at at least one device, information relating to error detection and measurements related to the error detection, the detection including errors reported by a user or detected by device software; receiving information sampled by the device at a model generator, the model generator including storage means containing information relating to device software and performance, and a model specifying device performance; in a model generator, adjusting the model based on the reported errors and associated measurements; sending the adjusted model to a device, where the device updates its software to correspond to the adjusted model; wherein the model generator adjusts the model by minimizing an error between an actual output of the device model and a desired output of the model based on a set of samples recorded on at least one electronic device, and the recorded samples include: Information about which of a number of predetermined states the device was in when the error occurred; Information about the time the error occurred, and Information about the number of errors at a specified time and / or state Includes:

[0067] Preferably, the predetermined states are transition-related and include a steady state, a pre-transition state, a transition state, and a post-transition state.

[0068] Preferably, the particular time includes the beginning of a state, the middle of a state, the end of a state, the entire state, or a combination thereof. Also, the count of errors includes a Boolean count and a tally count, or a duration ratio.

Claims

1. 1. A system for improving performance of at least one device, the device including at least one sensor and a model defining a response of the device in response to data generated by the at least one sensor; the system includes a model generator configured to analyze the data generated by the sensor and a response to the data and to register an error in the response compared to an intended response; the model generator is configured to adjust the model by minimizing an error between an actual output of the model of the device and a desired output of the model based on a set of samples recorded on the at least one device; The recorded sample is information regarding which of a number of predetermined states the device was in when the error occurred; Information about the time when the error occurred; and Information regarding the number of said errors at said specified time and / or said state Including, system.

2. The system of claim 1 , wherein the predetermined states are transition-related and include a steady state, a pre-transition state, a transition state, and a post-transition state.

3. The system of claim 1 , wherein the particular time includes a beginning of a state, a middle of a state, an end of a state, an entire state, or a combination thereof.

4. The system of claim 1 , wherein the count of errors includes a Boolean count and a tally count, or a duration ratio.

5. The system of claim 1 , wherein the model generator is a network system that communicates with multiple devices.

6. The system of claim 5 , wherein the model generator is adapted to communicate with a manufacturer that provides an initial model and expected performance of the device.

7. The system of claim 1 , wherein the device is provided with a use interface configured to receive error reports from a user of the device.

8. The system of claim 1 , wherein the error is registered as a measured deviation between an expected response of the device and the measured response.

9. 10. The system of claim 1, configured to analyze error detections and associated measurements in the sampled data based on a predetermined set of rules, and configured to iteratively reconfigure the model to reduce the number of error detections.

10. The system of claim 1 , wherein the sensor includes at least one of a camera, an acoustic sensor, an inertial sensor, a radar, or an optical measurement.

11. 1. A method for improving the performance of a particular type of device, comprising: sampling, in at least one device, information relating to error detection and measurements related to the error detection, said error detection including errors reported by a user or detected by software of said device; receiving the information sampled by the device in a model generator, the model generator including storage means containing information relating to the software and performance of the device, and a model specifying the performance of the device; adjusting the model in the model generator based on the reported errors and associated measurements; transmitting the adjusted model to the device, wherein the device updates the software to correspond to the adjusted model; Including, the model generator adjusts the model by minimizing an error between an actual output of the model of the device and a desired output of the model based on a set of samples recorded on the at least one device; The recorded sample is information regarding which of a number of predetermined states the device was in when the error occurred; Information about the time when the error occurred; and Information regarding the number of said errors at said specified time and / or said state Including, method.

12. The method of claim 11 , wherein the predetermined states are transition-related and include a steady state, a pre-transition state, a transition state, and a post-transition state.

13. The method of claim 11 , wherein the particular time includes the beginning of a state, the middle of a state, the end of a state, the entire state, or a combination thereof.

14. The method of claim 11 , wherein the count of errors includes a Boolean count and a tally count, or a duration ratio.

Images