Step recognition method and electronic device for supporting same

Abstract

An electronic device according to an embodiment may comprise at least one processor, a plurality of sensors, and a memory which is operatively connected to the at least one processor and the plurality of sensors and stores at least one instruction, wherein the at least one instruction is configured to cause, when executed individually or collectively by the at least one processor, the electronic device to: acquire first sensor information related to angular velocity and second sensor information related to acceleration through the plurality of sensors; detect a user's activity including a walking activity or a running activity on the basis of the first sensor information and the second sensor information; while the user's walking activity is detected, measure the number of steps of the user on the basis of the first sensor information; and while the user's running activity is detected, measure the number of steps of the user on the basis of the second sensor information.

Description

Step recognition method and electronic device supporting the same

[0001] The embodiments disclosed in this document relate to a step recognition method and an electronic device supporting the same.

[0002] With the development of digital technology, a variety of electronic devices capable of communication and personal information processing while on the move, such as mobile communication terminals, electronic notebooks, smartphones, tablet PCs, and wearable devices, are being released. These electronic devices have evolved from simple voice calls and message transmission functions to include various functions such as video calls, electronic notebook functions, document functions, email functions, internet functions, and camera functions.

[0003] In addition, the electronic device may provide health-related functions. For example, health-related functions may include a function to record information related to the user's activities in a health application. For instance, the electronic device may obtain at least one of the user's step count, step speed, distance traveled, or calorie consumption as activity information by utilizing information obtained through various sensors.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0005] An electronic device according to one embodiment disclosed in this document comprises at least one processor, a plurality of sensors, and a memory operatively connected to the at least one processor and the plurality of sensors and storing at least one instruction. When the at least one instruction is executed individually or collectively by the at least one processor, the electronic device may be configured to: acquire first sensor information related to angular velocity and second sensor information related to acceleration through the plurality of sensors; detect a user’s activity including walking or running activity based on the first sensor information and the second sensor information; measure the number of steps of the user based on the first sensor information while the user’s walking activity is detected; and measure the number of steps of the user based on the second sensor information while the user’s running activity is detected.

[0006] A method of operation of an electronic device according to an embodiment disclosed in this document may include: acquiring first sensor information related to angular velocity and second sensor information related to acceleration through a sensor; detecting a user’s activity, including walking or running activity, based on the first sensor information and the second sensor information; measuring the number of steps of the user based on the first sensor information while the user’s walking activity is detected; and measuring the number of steps of the user based on the second sensor information while the user’s running activity is detected.

[0007] A computer-readable storage medium according to one embodiment disclosed in this document may store at least one instruction that, when executed by a processor of an electronic device, causes the electronic device to: acquire first sensor information related to angular velocity and second sensor information related to acceleration through a sensor; detect a user’s activity including walking or running activity based on the first sensor information and the second sensor information; measure the number of steps of the user based on the first sensor information while the user’s walking activity is detected; and measure the number of steps of the user based on the second sensor information while the user’s running activity is detected.

[0008] A step recognition system according to one embodiment disclosed in this document comprises a first electronic device and a second electronic device, wherein the first electronic device comprises a first processor, a sensor, and a first memory operatively connected to the first processor and the sensor and storing at least one first instruction, wherein when the at least one first instruction is executed individually or collectively by the at least one first processor, the first electronic device is configured to: acquire first sensor information related to angular velocity and second sensor information related to acceleration through the sensor and provide to the second electronic device, wherein the second electronic device comprises a second processor and a second memory operatively connected to the second processor and storing at least one second instruction, wherein when the at least one second instruction is executed individually or collectively by the at least one second processor, the second electronic device is configured to: detect a user’s activity including walking activity or running activity based on the first sensor information and the second sensor information, and while the user’s walking activity is detected, measure the number of steps of the user based on the first sensor information, and while the user’s running activity is detected, the It can be configured to measure the number of steps taken by the user based on the second sensor information.

[0009] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0010] FIG. 1 shows a block diagram of an electronic device according to one embodiment.

[0011] FIGS. 2a and FIGS. 2b are drawings illustrating the form of an electronic device according to an embodiment.

[0012] FIG. 3 is a schematic diagram illustrating the configuration of an electronic device according to an embodiment.

[0013] FIG. 4 is a diagram illustrating a step recognition method of an electronic device according to an embodiment.

[0014] Figure 5 is a diagram illustrating a situation of misrecognition of steps in an electronic device according to an embodiment.

[0015] FIGS. 6a and 6b are drawings for explaining the step recognition operation of an electronic device according to an embodiment.

[0016] FIG. 7 is a diagram comparing the step recognition result of an electronic device according to a comparative example with the step recognition result of an electronic device according to one example.

[0017] FIG. 8 is a diagram illustrating the operation of an electronic device that detects user activity according to one embodiment.

[0018] Figure 9 is a diagram illustrating the input data of an activity input model.

[0019] Figure 10 is a diagram illustrating an activity recognition model according to an embodiment.

[0020] FIG. 11 is a diagram illustrating the feature map extraction operation of an activity recognition model according to one embodiment.

[0021] FIG. 12 is a diagram comparing the performance of a second computation layer composed of a lightweight computation layer according to one embodiment and the performance of a third computation layer composed of a lightweight computation layer.

[0022] FIGS. 13a to 13c are drawings for explaining an operation to improve the step recognition performance of an electronic device according to one embodiment.

[0023] FIGS. 14a and FIGS. 14b are drawings for explaining the step recognition operation of an electronic device according to one embodiment.

[0024] FIG. 15 is a flowchart illustrating the operation of an electronic device according to various embodiments.

[0025] FIG. 16 is a flowchart illustrating the step recognition operation of an electronic device according to various embodiments.

[0026] FIG. 17 is a block diagram of an electronic device in a network environment according to various embodiments.

[0027] Hereinafter, embodiments of the present invention are described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present invention.

[0028]

[0029] FIG. 1 is a block diagram of an exemplary electronic device (100) capable of performing the operations described in this document.

[0030] Referring to FIG. 1, the electronic device (100) may be one of various forms of electronic devices, such as a notebook (190), smartphones (191) having various form factors (e.g., a bar-type smartphone (191-1), a foldable-type smartphone (191-2), or a sliderable (or rollable)-type smartphone (191-3)), a tablet (192), a cellular phone (not shown), and other similar computing devices (not shown). The components, their relationships, and their functions illustrated in FIG. 1 are illustrative only and are not intended to limit the implementations described or claimed herein. The electronic device (100) may be referred to as a mobile device, a user device, a multifunction device, a portable device, or a server.

[0031] The electronic device (100) may include components comprising at least one processor (110) (hereinafter referred to as processor (110)), at least one memory (120) (hereinafter referred to as memory (120)), at least one display (140) (hereinafter referred to as display (140)), at least one image sensor (150) (hereinafter referred to as image sensor (150)), at least one communication circuit (160) (hereinafter referred to as communication circuit (160)), and / or at least one sensor (170) (hereinafter referred to as sensor (170)). The components are merely exemplary. For example, the electronic device (100) may include other components (e.g., power management integrated circuitry (PMIC), audio processing circuit, antenna, rechargeable battery, or input / output interface). For example, some components may be omitted from the electronic device (100). For example, some components may be integrated into a single component.

[0032] The processor (110) may be implemented as one or more integrated circuit (or circuitry) chips and may perform various data processing operations. The processor (110) may include at least one electrical circuit and may process instructions (or programs, data) stored in memory (120) individually or collectively in a distributed manner. The processor (110) may include a processor assembly comprising one or more processing circuits. The processor (110) may include any processing circuit that is operative to control the performance and operations of one or more components of the electronic device (100) (e.g., memory (120), display (140), image sensor (150), communication circuit (160), and / or sensor (170)). For example, the processor (110) (e.g., application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). For example, the processor (110) may be implemented with a plurality of cores (or at least one core circuit), a plurality of chips, or a plurality of chipsets. For example, the processor (110) may include one or more processing circuits. For example, the processor (110) may include one or more processing circuits configured to perform the various functions of the present disclosure individually and / or collectively. As an example without limitation, at least a portion of the processor (110) may be included in a first chip of the electronic device (100), and at least another portion of the processor (110) may be included in a second chip of the electronic device (100) different from the first chip of the electronic device (100).

[0033] For example, the processor (110) may include a central processing unit (111), a graphics processing unit (112), a neural processing unit (113), an image signal processor (114), a display controller (115), a memory controller (116), a storage controller (117), a communication processor (118), and / or a sensor interface (119). These components of the processor (110) are merely exemplary. For example, the processor (110) may include other components. For example, some components of the processor (110) may be omitted from the processor (110). For example, some components of the processor (110) may be included as separate components of the electronic device (100) outside the processor (110). For example, some components of the processor (110) (e.g., memory controller (116)) may be included within other components (e.g., at least part of memory (120), an interface (e.g. available for connection to at least one component of the electronic device (100)), a display (140) and / or an image sensor (150)).

[0034] The processor (110) may cause other components of the electronic device (100) to perform various operations by executing instructions stored in memory (120). The CPU (111) (or central processing circuit) may be configured to control the components of the processor (110) based on the execution of instructions stored in memory (120) (e.g., volatile memory (121) and / or non-volatile memory (122)). The GPU (112) (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (113) (or neural processing circuit, or AI (artificial intelligence) chip) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). An ISP (114) (or image signal processing circuit) may be configured to process a raw image acquired through an image sensor (150) into a format suitable for a component within the electronic device (100) or a component of the processor (110). A display controller (115) (or display control circuit, or DPU (display processing unit)) may be configured to process an image acquired from a CPU (111), GPU (112), ISP (114), or memory (120) (e.g., volatile memory (121)) into a format suitable for a display (140). A memory controller (116) (or memory control circuit) may be configured to control reading data from the volatile memory (121) and writing data to the volatile memory (121). A storage controller (117) (or storage control circuit) may be configured to control reading data from the non-volatile memory (122) and writing data to the non-volatile memory (122).The CP (118) (communication processing circuit) may be configured to process data obtained from a component of the processor (110) into a format suitable for transmitting to another electronic device via the communication circuit (160), or to process data obtained from another electronic device via the communication circuit (160) into a format suitable for processing by the component of the processor (110). For example, the communication circuit (160) may include one or more communication circuits. The sensor interface (119) (or sensing data processing circuit, sensor hub) may be configured to process data regarding the state of the electronic device (100) and / or the state around the electronic device (100), obtained through the sensor (170), into a format suitable for the component of the processor (110). The processor (110) may control the operations of the electronic device (100) by executing instructions stored in the memory (120). For example, the processor (220) may correspond to a plurality of processors that divide and collectively perform a plurality of operations among the processors.

[0035] Memory (120) may include one or more storage media (or one or more storage devices). For example, memory (120) may include a memory assembly comprising one or more storage media. For example, the one or more storage media may include a hard drive, a permanent memory such as flash memory, read-only memory (ROM) (e.g., non-volatile memory (122)), a semi-permanent memory such as random access memory (RAM) (e.g., volatile memory (121)), any other suitable type of storage (or storage assembly), or any combination thereof. Memory (120) may include a cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (100). As an example not limited to, the cache memory may be included within the processor (110). The memory (120) may be fixedly embedded within the electronic device (100) or incorporated into one or more suitable types of components (e.g., a SIM (subscriber identity module) card and / or an SD (secure digital) card) that can be repeatedly inserted into and removed from the electronic device (100).

[0036] For example, memory (120) may store one or more software applications, such as operating system (or system) software applications, firmware software applications, driver software applications, plugin (e.g., add-in, add-on, and / or applet) software applications, and / or any other suitable software applications. For example, the one or more software applications may include instructions executable by the processor (110). For example, memory (120) may store instructions that can be called by an application programming interface (API). For example, memory (120) may store instructions within a library.

[0037]

[0038] An electronic device (100) according to one embodiment may provide information related to a user's activity through an application (e.g., a health application). For example, as part of the operation of providing information related to a user's activity, the electronic device (100) may recognize a user's steps using information obtained through a sensor (170). This will be explained in detail through FIGS. 2a to 16 below. In addition, at least one of the embodiments described through FIGS. 2a to 16 below may be combined with other embodiments.

[0039]

[0040] FIGS. 2a and FIGS. 2b are drawings illustrating the form of an electronic device according to an embodiment.

[0041] Referring to FIGS. 2a and 2b, the electronic device (200) (e.g., the electronic device (100) of FIG. 1) may be a wearable device in the form of a watch worn on a part of the body (e.g., the wrist). However, this is merely one embodiment, and depending on the embodiment, the electronic device (200) may be implemented as a wearable device in various forms, such as a band or a ring.

[0042] According to one embodiment, the electronic device (200) may be implemented in various shapes. For example, the electronic device (200) may be implemented in a circular shape. However, this is merely one embodiment, and depending on the embodiment, the electronic device (200) may be implemented in various shapes such as a rectangle, a square, an ellipse, etc. Additionally, the electronic device (200) may be implemented in a shape having a curved surface for the user to grip.

[0043] The aforementioned electronic device (200) may include various components. According to one embodiment, the various components may be at least some of the components of the electronic device (100) shown in FIG. 1. In this regard, a detailed explanation will be provided with reference to FIG. 3 below.

[0044]

[0045] FIG. 3 is a schematic diagram illustrating the configuration of an electronic device according to an embodiment.

[0046] Referring to FIGS. 2a to 3, an electronic device (200) according to one embodiment may be composed of at least one housing (201) (hereinafter referred to as housing (201)), at least one sensor module (210) (or sensor circuit (210)) (hereinafter referred to as sensor module (210)), at least one display (220) (hereinafter referred to as display (220)), at least one memory (230) (hereinafter referred to as memory (230)), and at least one processor (240) (hereinafter referred to as processor (240)).

[0047] The components of the electronic device (200) described above are merely one embodiment, and depending on the embodiment, the electronic device (200) may be implemented to have more components than the components shown in FIG. 3 or fewer components.

[0048] For example, at least some of the components of the electronic device (100) illustrated in FIG. 1 (e.g., image sensor (150) and / or communication circuit (160)) may be included in the configuration of the electronic device (200). Additionally, at least some components of the electronic device (100) that are not illustrated in FIG. 1 but may be included in the configuration of the electronic device (200) (e.g., battery and / or antenna module) may be included in the configuration of the electronic device (200). Additionally or optionally, at least one component of the electronic device (200) may be integrated with another component. For example, the electronic device (200) may correspond to the electronic device (1701) of FIG. 17.

[0049] According to one embodiment, the housing (201) may form the exterior of the electronic device (200). For example, the housing (201) may include a first surface (e.g., front), a second surface (e.g., rear), and a third surface (e.g., side) that surrounds the space between the first surface and the second surface.

[0050] The housing (201) may provide space for mounting components of the electronic device (200) (e.g., sensor module (210), display (220), memory (230) and processor (240)). According to one embodiment, some of the components of the electronic device (200) mounted inside the housing (201) (e.g., sensor module (210) and / or display (220)) or at least some of them may be exposed through at least a part of the housing (201).

[0051] According to one embodiment, the sensor module (210) (e.g., the sensor (170) shown in FIG. 1) may include at least one first sensor (211) (hereinafter referred to as the first sensor (211)) configured to acquire biometric information.

[0052] According to one embodiment, the first sensor (211) can acquire a pulse wave signal with respect to the test part (or test area) (e.g., wrist). The pulse wave signal may be a PPG signal. For example, the first sensor (211) may be composed of at least one light-emitting part (211-1) (hereinafter referred to as the light-emitting part (211-1)) and at least one light-receiving part (211-2) (hereinafter referred to as the light-receiving part (211-2)).

[0053] According to one embodiment, the light-emitting part (211-1) may be composed of a first light-emitting element that irradiates light of a first wavelength (e.g., red (wavelength: 600 nm - 700 nm)) onto a test part and a second light-emitting element that irradiates light of a second wavelength (e.g., infrared (wavelength: 780 nm - 1000 μm)) onto a test part. For example, the first light-emitting element and the second light-emitting element constituting the light-emitting part (211-1) may be implemented using a light-emitting diode, an organic light-emitting diode, quantum dot light-emitting diodes, a laser diode, or a phosphor.

[0054] However, this is merely one example, and the embodiments disclosed in this document are not limited thereto. For example, the light-emitting part (211-1) may further include one or more additional light-emitting elements having a wavelength that is the same as or different from the wavelength of the first light-emitting element and the second light-emitting element (e.g., blue wavelength, green wavelength, etc.).

[0055] According to one embodiment, the light receiving unit (211-2) can receive light and convert the received light into a current signal by photoelectric conversion. For example, the light receiving unit (211-2) may include at least one light receiving element for detecting light of a first wavelength irradiated from a first light-emitting element and light of a second wavelength irradiated from a second light-emitting element. For example, at least one light receiving element may include a photo detector or a photo diode.

[0056] According to an embodiment, the light receiving unit (211-2) can detect light reflected from the inspection unit by being irradiated from the light emitting unit (211-1) (e.g., a first light emitting element and a second light emitting element). In this regard, the light receiving unit (211-2) and the light emitting unit (211-1) may be placed on the same surface as each other.

[0057] According to an embodiment, the light receiving unit (211-2) can detect light that is irradiated from the light emitting unit (211-1) and passes through the inspection unit. In this regard, the light receiving unit (211-2) and the light emitting unit (211-1) may be arranged to face each other.

[0058] However, this is merely one example, and the embodiments disclosed in this document are not limited thereto. For example, the light receiving part (211-1) and the light emitting part (211-2) may be arranged in various forms. Additionally, according to the embodiment, at least a portion of the first sensor (211) may be exposed through a portion of the second surface (e.g., rear) of the housing (201).

[0059] According to one embodiment, the sensor module (210) may include at least one second sensor (212) (hereinafter referred to as the second sensor (212)) configured to acquire inertial information related to the attitude and / or movement of the electronic device (200). The inertial information acquired through the second sensor (212) may include acceleration information (e.g., 3-axis acceleration information) and angular velocity information (e.g., 3-axis angular velocity information). For example, the second sensor (212) may be composed of at least one of an acceleration sensor, a gyroscope, a gesture sensor, or a geomagnetic sensor.

[0060] The configuration of the sensor module (210) described above is one embodiment, and various embodiments are not limited thereto. Depending on the embodiment, the sensor module (210) may be implemented with more sensors than the sensors described above. For example, at least one electrode sensor connected to the first sensor (211) and configured to acquire bio-information about a part of the body, and / or at least one body temperature sensor configured to acquire temperature information about a part of the body (e.g., wrist) in a contact manner or in a non-contact manner may be added to the configuration of the sensor module (210).

[0061] According to one embodiment, a display (220) (e.g., the display (140) shown in FIG. 1) may be used to provide information processed by an electronic device (200). For example, the display (220) may display a screen or user interface for information processed by the electronic device (200). According to an embodiment, the display (200) may be exposed through a portion of a first surface (e.g., the front) of the housing (201).

[0062] According to one embodiment, the memory (230) (e.g., the memory (120) illustrated in FIG. 1) can store various data used by components of the electronic device (200). For example, the memory (230) can store instructions that cause the electronic device (200) to perform functions (e.g., operations).

[0063] According to one embodiment, a processor (240) (e.g., the processor (110) shown in FIG. 1) may be operatively connected to a sensor module (210), a display (220), and a memory (230), and may control various components (e.g., hardware or software components) of an electronic device (200).

[0064] For example, the processor (240) may support health-related functions. Specific embodiments thereof will be described in more detail with reference to FIG. 4. Furthermore, the description with reference to FIG. 4 is merely one embodiment, and various known technologies may be taken into account as embodiments described in this document.

[0065]

[0066] FIG. 4 is a diagram illustrating a step recognition method of an electronic device according to an embodiment. FIG. 5 is a diagram illustrating a situation of misrecognition of steps by an electronic device.

[0067] Referring to FIG. 4, an electronic device (200) (e.g., processor (240)) can recognize (or detect) a user's steps using information obtained through a sensor module (210) as part of an operation that supports health-related functions. For example, the electronic device (200) can provide various information related to the user's steps through an application (e.g., a health application).

[0068] According to one embodiment, a step may be a movement of stepping with two feet. For example, an electronic device (200) may recognize at least one of a step width (e.g., stride length), the number of steps, or a step speed.

[0069] In this regard, the electronic device (200) can utilize inertial information obtained through the second sensor (212) for step recognition. The inertial information used for step recognition may be acceleration information related to the first type of arm movement (e.g., 3-axis acceleration information). For example, the first type of arm movement may be referenced as the movement of an arm that swings naturally like a pendulum motion while walking (e.g., swing movement).

[0070] For example, as illustrated in 410 of FIG. 4, the electronic device (200) may be worn on a part of the body (401) (e.g., wrist), and the inertial information obtained by the second sensor (212) may be acceleration information indicating a change in magnitude and / or direction of velocity for the arm (403) swinging during walking. However, this is merely one embodiment, and the embodiments disclosed in this document are not limited thereto. For example, angular velocity information (e.g., 3-axis angular velocity information) obtained through the second sensor (212) may be used for step recognition (e.g., step count measurement).

[0071] According to one embodiment, the electronic device (200) can utilize inertial information satisfying specified conditions for step recognition when recognizing a step.

[0072] According to an embodiment, the specified condition may include a first specified condition associated with a threshold range (430) determined by a maximum threshold value (Tmax) (424) (e.g., approximately 5 m / s²) and a minimum threshold value (Tmin) (425) (e.g., approximately 2 m / s²), as illustrated in 420 of FIG. 4.

[0073] In this regard, the electronic device (200) can extract peaks (426-1, 426-2, and 426-3) of inertial information included within the threshold range (430) as inertial information satisfying the first specified condition. For example, the electronic device (200) can calculate the number of steps (e.g., cumulative number of steps) based on the number of inertial information satisfying the first specified condition extracted over a certain period of time. Additionally, the electronic device (200) can recognize peaks (426-5) of inertial information outside the threshold range (430) as inertial information that does not satisfy the first specified condition, and exclude them from being used for step recognition.

[0074] According to an embodiment, the specified condition may include a second specified condition related to the minimum time interval (440) between the peaks (426-1) and (426-2) of the inertia information, as illustrated in 420 of FIG. 4.

[0075] In this regard, the electronic device (200) can extract a peak among the peaks (426-1, 426-2 and 426-3) of the inertial information that satisfies the minimum time interval (440) as inertial information that satisfies the second specified condition, and use this to recognize steps. For example, the electronic device (200) can recognize peaks included within the minimum time interval (440) as inertial information that does not satisfy the second specified condition, and exclude them from step recognition.

[0076] Additionally or optionally, the electronic device (200) may extract a peak among the peaks (426-1, 426-2 and 426-3) of the inertial information that satisfies both the first specified condition and the second specified condition, and use these to recognize steps.

[0077] As described above, an electronic device (200) according to one embodiment can recognize a step using acceleration information (e.g., 3-axis acceleration information).

[0078] However, the characteristics of the acceleration information obtained through the second sensor (212) may vary depending on the user's activity. For example, the user's activity may include at least walking and running activities.

[0079] Generally, the user can perform walking activities by alternating steps with their left and right feet. In addition, during the walking activity, the user's arms can swing naturally like a pendulum.

[0080] For example, as shown in 510 of FIG. 5, when the left foot (or right foot) is moved forward, the right arm (or left arm) can also move toward the front of the torso. In other words, when the left foot (or right foot) is moved forward, the left arm (or right arm) can move toward the back of the torso.

[0081] In addition, as shown in 520 of FIG. 5, when moving the right foot (or left foot) forward again, the left arm (or right arm) can also move toward the front of the torso. In other words, when moving the right foot (or left foot) forward, the right arm (or left arm) can move toward the back of the torso.

[0082] Accordingly, the magnitude of the peak acceleration information obtained for the arm moved toward the rear of the torso may have a relatively small value, and the magnitude of the peak acceleration information obtained for the arm moved toward the front of the torso may have a relatively large value.

[0083] For example, in the case where the arm (e.g., the arm on which the electronic device (200) is worn) moves toward the rear of the torso during the user's walking activity (hereinafter referred to as the first situation during walking), acceleration information in which the peak is smaller than the minimum threshold (Tmin) (425) can be obtained through the second sensor (212). Additionally, in the case where the arm moves toward the front of the torso during the user's walking activity (hereinafter referred to as the second situation during walking), acceleration information in which the peak is included within the threshold range (430) can be obtained through the second sensor (212).

[0084] In this situation, the electronic device (200) can normally recognize the occurrence of a step at the point in time (531) corresponding to the second situation during walking shown in 530 of FIG. 5, but may misrecognize the occurrence of a step at the point in time (533) corresponding to the first situation during walking.

[0085] In this regard, the electronic device (200) according to one embodiment can further improve step recognition performance by selecting acceleration information or angular velocity information according to the user's activity and utilizing it for step recognition. Specific embodiments thereof will be described in detail through FIGS. 6a and 6b below.

[0086]

[0087] FIGS. 6a and FIGS. 6b are drawings for explaining the step recognition operation of an electronic device according to an embodiment.

[0088] Referring to FIGS. 6a and 6b, an electronic device (200) (e.g., processor (240)) can improve step recognition performance by using different sensor information depending on the user's activity. For example, when the user's first activity is detected (or identified) (e.g., while the first activity is detected), the electronic device (200) can select one of acceleration information or angular velocity information that is not affected by the first activity and use it for step recognition. Additionally, when the user's second activity is detected (or identified) (e.g., while the second activity is detected), the electronic device (200) can select the other of acceleration information or angular velocity information that is not affected by the second activity and use it for step recognition.

[0089] According to one embodiment, the user's first activity may be a walking activity. As described above through 510 and 520 of FIG. 5, when acceleration information is used for step recognition, a point in time (533) corresponding to a first situation during walking (e.g., a situation where the arm moves toward the back of the torso during the user's walking activity) may be misidentified as no step occurring. Accordingly, while the user's first activity is detected (or identified), the electronic device (200) may use first sensor information (e.g., 3-axis angular velocity information) that is relatively unaffected by arm movement for step recognition.

[0090] In this regard, as illustrated in FIG. 6a, based on a first detection result (610) indicating a first activity of a user, the processor (240) (or electronic device (200)) may instruct (612) the sensor module (210) (e.g., a second sensor (212)) to provide first sensor information. Accordingly, the processor (240) (or electronic device (200)) may recognize a step based on the first sensor information (614) provided by the sensor module (210).

[0091] According to one embodiment, the user's second activity may be a running activity. In a situation where a running activity is detected, the change in speed of the arm movement may be greater than in a situation where a walking activity is detected, but the change in the angle of the arm movement may not be relatively large. Accordingly, while the user's second activity is detected (or identified), the electronic device (200) may utilize second sensor information (e.g., 3-axis acceleration information) that sufficiently reflects the change in speed of the arm movement for step recognition.

[0092] In this regard, as illustrated in FIG. 6b, based on a second detection result (620) indicating a second activity of the user, the processor (240) (or electronic device (200)) may instruct (622) the sensor module (210) (e.g., a second sensor (212)) to provide second sensor information. Accordingly, the processor (240) (or electronic device (200)) may recognize a step based on the second sensor information (624) provided by the sensor module (240).

[0093] Additionally or optionally, an electronic device (200) (e.g., memory (230)) may store a step recognition model (231) configured to recognize a user's step based on inertial information obtained through a second sensor (212). In this case, the electronic device (200) may recognize a step using the step recognition model (231). For example, the step recognition model (231) may be an artificial intelligence model generated through machine learning and may include a plurality of artificial neural network layers. For example, the step recognition model (231) may take first sensor information (e.g., 3-axis angular velocity information) and / or second sensor information (e.g., 3-axis acceleration information) as inputs and output a step recognition result.

[0094]

[0095] FIG. 7 is a diagram comparing the step recognition result of an electronic device according to a comparative example with the step recognition result of an electronic device according to one example.

[0096] 710 in FIG. 7 represents the result of step recognition of an electronic device according to a comparative embodiment. For example, the electronic device according to the comparative embodiment can utilize acceleration information obtained in a situation where walking activity is detected for step recognition. It can be confirmed that the electronic device according to the comparative embodiment misidentified some acceleration information as no step occurring. For example, as described above through 510 and 520 in FIG. 5, the electronic device according to the comparative embodiment can misidentify as no step occurring at a point in time (533) corresponding to the first situation during walking.

[0097] Figure 720 of FIG. 7 represents the result of step recognition of an electronic device (200) according to one embodiment. For example, the electronic device (200) according to one embodiment can utilize angular velocity information for step recognition in situations where walking activity is detected. In this way, the electronic device (200) according to one embodiment can confirm that it has successfully recognized the occurrence of a step at a point in time (533) corresponding to a first situation during walking and at a point in time (531) corresponding to a second situation during walking.

[0098] As can be seen from FIG. 7, the electronic device (200) according to one embodiment can improve step recognition performance by utilizing angular velocity information that is relatively unaffected by arm movement during the detection of a first activity of the user.

[0099]

[0100] FIG. 8 is a diagram illustrating the operation of an electronic device for detecting user activity according to one embodiment. FIG. 9 is a diagram illustrating input data of an activity input model.

[0101] Referring to FIG. 8, an electronic device (200) (e.g., processor (240)) may utilize an artificial intelligence model generated through machine learning to recognize user activity. In this regard, an activity recognition model (232) configured to recognize (or detect) user activity based on inertial information obtained through a second sensor (212) may be stored in the electronic device (200) (e.g., memory (230)) or externally (e.g., server).

[0102] According to one embodiment, the activity recognition model (232) can take information obtained through the sensor module (210) as input and output an activity recognition result (830) for the user. For example, first sensor information (810) (e.g., 3-axis angular velocity information) and second sensor information (820) (e.g., 3-axis acceleration information) obtained through the second sensor (212) can be provided as inputs to the activity recognition model (232).

[0103] According to an embodiment, the electronic device (200) can input the first sensor information (810) and the second sensor information (820) into an activity recognition model (232) by concatenating them.

[0104] In this regard, as illustrated in FIG. 9, the electronic device (200) may perform preprocessing (930) on the first sensor information (810) and the second sensor information (820) when inputting the first sensor information (810) and the second sensor information (820) into the activity recognition model (232). According to one embodiment, the preprocessed first sensor information (810) may be angular velocity information measured over a certain period of time. Additionally, the preprocessed second sensor information (820) may be acceleration information measured over a certain period of time.

[0105] For example, the electronic device (200) can perform preprocessing (930) on the first sensor information (810) by dividing the angular velocity information (910) obtained through the second sensor (212) into fixed time intervals. Additionally, the electronic device (200) can perform preprocessing (930) on the second sensor information (820) by dividing the acceleration information (920) obtained through the second sensor (212) into fixed time intervals. Additionally or optionally, the electronic device (200) may obtain refined angular velocity information and / or acceleration information by processing in a manner such as filtering and sampling, and divide them into fixed time intervals.

[0106] The aforementioned activity recognition model (232) may include a plurality of artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, a transformer network, or a combination of two or more of these, but is not limited to the examples described above. Additionally, the activity recognition model (232) may include a hardware structure in addition to or substantially in addition to the software structure. Specific embodiments thereof will be described in detail through FIG. 10 below.

[0107]

[0108] FIG. 10 is a diagram illustrating an activity recognition model according to one embodiment. FIG. 11 is a diagram illustrating a feature map extraction operation of an activity recognition model according to one embodiment.

[0109] Referring to FIG. 10, the activity recognition model (232) may be composed of an input layer (1010), a convolutional layer (1020), a fully connected layer (1030), and an output layer (1040).

[0110] According to one embodiment, the input layer (1010) may perform the role of receiving input data (e.g., first sensor information (810) and second sensor information (820)). According to an embodiment, the input layer (1010) may perform preprocessing on the input data and provide the preprocessed input data to the convolutional layer (1020).

[0111] According to one embodiment, the convolutional layer (1020) can extract features from input data provided from the input layer (1010) and generate a feature map based thereon. The feature map is a matrix generated as a result of a convolution operation and may contain features of the input data. For example, the convolutional layer (1020) can generate a feature map by scanning the input data in a specified manner using at least one filter (or kernel) having a certain size. This feature map can be provided as an input to the entire connection layer (1030).

[0112] According to one embodiment, the entire connection layer (1030) can perform a final prediction based on the features of the input data (e.g., a feature map). For example, the entire connection layer (1030) can classify the user's activities based on the feature map extracted from the convolutional layer (1020) after converting it into a one-dimensional vector.

[0113] According to one embodiment, the output layer (1040) can receive information transmitted from the entire connection layer (1030) and output a final result. For example, the output layer (1040) can output a final result based on information learned by the activity recognition model (232).

[0114] Generally, the performance of the activity recognition model (232) may be related to the number of feature maps (e.g., number of channels or depth). Accordingly, the activity recognition model (232) can maintain a certain level of performance by generating a certain number of feature maps.

[0115] In this regard, according to one embodiment, the convolutional layer (1020) may be composed of a plurality of computational layers (e.g., a first computational layer (1021) to a fourth computational layer (1027)) each configured to generate a certain number of feature maps, as shown in FIG. 10. However, this is merely one embodiment, and depending on the embodiment, more or fewer computational layers may be included in the convolutional layer (1020).

[0116] For example, the convolutional layer (1020) can generate feature maps whose size gradually decreases and whose number gradually increases through operations of the first operation layer (1021) to the third operation layer (1025). Additionally, to solve the problem of increased computational load caused by inputting many feature maps into the entire connection layer (1030), the convolutional layer (1020) can reduce the number of specific maps to a certain level through the fourth operation layer (1027).

[0117] For example, the first operation layer (1021) can generate a first feature map having a first size and a first number (e.g., 16) using input data and a first filter. Additionally, the second operation layer (1023) can generate a second feature map having a second size smaller than the first size and a second number greater than the first number (e.g., 32) using the first feature map and a second filter. Additionally, the third operation layer (1025) can generate a third feature map having a third size smaller than the second size and a number equal to the second number (or a third number greater than the second number) using the second feature map and a third filter. Additionally, the fourth operation layer (1027) can generate a fourth feature map having a fourth size smaller than the third size and a number smaller than the second number or a number smaller than the third number (e.g., 16) using the third feature map and a fourth filter.

[0118] Additionally or optionally, the first operation layer (1021) to the fourth operation layer (1027) may further include a pooling layer (e.g., a max pooling layer) to further reduce the size of the generated feature map.

[0119] As described above, the convolutional layer (1020) may use at least one filter when generating a feature map. As the number of such filters increases, the performance of the activity recognition model (232) improves, but there is a problem that the amount of computation of the activity recognition model (232) (e.g., the entire connected layer (1030)) increases.

[0120] To solve the above problems, some of the plurality of computational layers (e.g., first computational layer (1021) to fourth computational layer (1027)) according to one embodiment (e.g., first computational layer (1021), third computational layer (1025) and fourth computational layer (1027)) may be configured as general computational layers, and other parts (e.g., second computational layer (1021)) may be configured as lightweight computational layers.

[0121] For example, a general computation layer may be a computation layer that uses the same number of filters as the number of feature maps to be generated. Additionally, a lightweight computation layer may be a computation layer that uses fewer filters than the number of feature maps to be generated.

[0122] In this regard, the first operation layer (1021), composed of a general operation layer, may use input data (1111) and a first number (e.g., 16) of first filters (1113) to generate a first feature map (1115) (e.g., 16 first feature maps), as illustrated in 1110 of FIG. 11. Additionally, the third operation layer (1025), composed of a general operation layer, may use a third number (e.g., 32) of third filters to generate a third feature map (e.g., 32 third feature maps), and the fourth operation layer (1027), composed of a general operation layer, may use a fourth number (e.g., 16) of fourth filters to generate a fourth feature map (e.g., 16 fourth feature maps).

[0123] However, the second operation layer (1023) composed of a lightweight operation layer may only use a second number of filters (e.g., 16) that is smaller than the number (e.g., 32) required to generate a second feature map (1125) (e.g., 32 second feature maps), as shown in 1120 of FIG. 11.

[0124] For example, the second operation layer (1023) can use 16 first feature maps (1115) generated in the previous operation layer (e.g., the first operation layer (1021)) as input data and use a second number (e.g., 16) of second filters (1121) to generate the second feature maps (1125).

[0125] In this case, the second operation layer (1023) can generate a first intermediate feature map (1123) (e.g., 16 first intermediate feature maps) using input data (e.g., a first feature map) (1115) and a second filter (1123). Additionally, the second operation layer (1023) can generate a second intermediate feature map (1124) by copying (1131) the first feature map (1115) generated in the previous operation layer (e.g., the first operation layer (1021)), and generate a second feature map (1225) (e.g., a total of 32 second feature maps) by merging the first intermediate feature map (1123) and the second intermediate feature map (1124).

[0126] As described above, the activity recognition model (232) according to one embodiment can improve recognition performance while reducing the amount of computation of the activity recognition model (232) by configuring the second computation layer among the plurality of computation layers as a lightweight computation layer. However, this is merely one embodiment, and the embodiments described in this document are not limited thereto. For example, the activity recognition model (232) according to one embodiment may configure a computation layer other than the second computation layer among the plurality of computation layers as a lightweight computation layer.

[0127] However, since the lightweight computation layer uses a feature map (e.g., first feature map (1115)) generated in a previous computation layer (e.g., first computation layer (1021)) as input, it is necessary to configure a different computation layer other than the first computation layer (1021) located at the very front of the convolutional layer (1020) as the lightweight computation layer. Additionally, since the fourth computation layer (1027) located at the very end of the convolutional layer (1020) reduces the number of specific maps to a certain level and provides them to the entire connection layer (1030), it is necessary to configure a different computation layer other than the fourth computation layer (1027) as the lightweight computation layer.

[0128] Accordingly, the activity recognition model (232) according to one embodiment may improve recognition performance while reducing the amount of computation of the activity recognition model (232) by configuring the second computation layer (1023) and / or the third computation layer (1025) among the plurality of computation layers as lightweight computation layers.

[0129]

[0130] FIG. 12 is a diagram comparing the performance of a second computation layer composed of a lightweight computation layer according to one embodiment and the performance of a third computation layer composed of a lightweight computation layer.

[0131] Referring to FIG. 12, it can be seen that the performance (1210) of the second operation layer (1023) composed of a lightweight operation layer is superior to the performance (1220) of the third operation layer (1025) composed of a lightweight operation layer.

[0132] For example, the number of trainable parameters, multiply-accumulate (MAC) and interface time, which are indicators measuring the computational amount of the activity recognition model (232), can be confirmed to be superior when the second computation layer (1023) is configured as a lightweight computation layer compared to when the third computation layer (1025) is configured as a lightweight computation layer.

[0133] However, although the difference in accuracy is not significant, it can be confirmed that configuring the third operation layer (1025) as a lightweight operation layer is superior to configuring the second operation layer (1023) as a lightweight operation layer.

[0134] This may mean that the recognition performance of the activity recognition model (232) can be further improved when the second operation layer (1023) among the multiple operation layers is configured as a lightweight operation layer (e.g., compared to when the third operation layer (1025) is configured as a lightweight operation layer).

[0135]

[0136] FIGS. 13a to 13c are drawings for explaining an operation to improve the step recognition performance of an electronic device according to one embodiment.

[0137] An electronic device (200) according to one embodiment can further improve step recognition performance by selecting acceleration information or angular velocity information according to the user's activity and utilizing it for step recognition.

[0138] However, a second type of arm movement may occur during the user's activity. Inertial information reflecting this second type of movement may cause a decrease in step recognition performance. For example, the second type of arm movement that causes a decrease in step recognition performance may include at least one of the movements (1301), (1341) of raising the arm to check the electronic device (200) worn on the body, the movements (1303), (1343) of raising the arm to wipe sweat, or the arm movements (1305), (1345) for warm-up exercises, as illustrated in FIG. 13a and FIG. 13b.

[0139] For example, in a non-gait state (e.g., stationary state) and in a situation where the second type of arm movement is not accompanied, inertial information that does not satisfy the specified conditions can be obtained through the second sensor (212). For example, inertial information in which the peak is smaller than the minimum threshold (Tmin) (425) can be obtained through the second sensor (212), and the electronic device (200) can normally recognize that no step has occurred by using this inertial information.

[0140] However, in a non-gait state accompanied by a second type of arm movement, inertial information satisfying specified conditions can be obtained through the second sensor (212). For example, as shown in 1330 of FIG. 13a, the peak (1331) of the inertial information reflecting the second type of arm movement may be included within a threshold range (430), and the processor (240) may use this inertial information to misidentify it as a step.

[0141] In this regard, the electronic device (200) may stop the operation of recognizing steps (e.g., measuring the number of steps) in a situation where it is in a non-walking state and accompanied by a second type of arm movement (1301), (1303) and / or (1305). For example, the electronic device (200) may prevent misrecognition of a step occurring in a non-walking state by stopping the step recognition operation until a walking state is detected.

[0142] In addition, a decrease in gait recognition performance may also be caused by a second type of arm movement accompanying the gait state.

[0143] For example, in a situation where walking is in progress and the second type of arm movement is not accompanied, inertial information (e.g., angular velocity information) that does not satisfy the specified conditions can be obtained through the second sensor (212). For example, inertial information in which the highest point is included within the threshold range (430) can be obtained through the second sensor (212), and the processor (240) can use this inertial information to normally recognize the occurrence of a step.

[0144] However, as illustrated in 1350 of FIG. 13b, some of the inertial information reflecting the accompanying second type of arm movements (1341), (1343) and / or (1345) in the walking state may be outside the threshold range (430), and the processor (240) may use this inertial information to misidentify that no steps have occurred.

[0145] In this regard, the electronic device (200) according to one embodiment can prevent the occurrence of the aforementioned problem by recognizing the step based on the magnitude of the angular velocity in the frequency domain when the device is in a walking state and accompanied by a second type of arm movement.

[0146] For example, the electronic device (200) may store a reference step frequency associated with a walking state that is not accompanied by a second type of arm movement. Additionally, when the electronic device (200) detects a situation in which a walking state is present and accompanied by a second type of arm movement, it may convert the angular velocity magnitude information in the time domain into the frequency domain. For example, the electronic device (200) may recognize a step based on information corresponding to a reference frequency (e.g., reference step frequency) among the information in the frequency domain (e.g., information shown in FIG. 13c). In one embodiment, the electronic device (200) may use information corresponding to a designated reference frequency among the information in the frequency domain for the user's step recognition (e.g., measuring the number of steps). Additionally, the electronic device (200) may exclude information in the frequency domain that does not correspond to a designated reference frequency from the user's step recognition (e.g., measuring the number of steps).

[0147]

[0148] FIG. 14a is a drawing for explaining the step recognition operation of an electronic device according to one embodiment.

[0149] Referring to FIG. 14a, an electronic device (200) (e.g., processor (240)) can recognize the user's activity and the user's steps using inertial information obtained by a sensor module (210) (e.g., second sensor (212)). For example, first sensor information (810, 1401) (e.g., 3-axis angular velocity information) and second sensor information (820, 1402) (e.g., 3-axis acceleration information) can be used to recognize the user's activity and the user's steps.

[0150] According to one embodiment, inertial information used to recognize the user's activity and the user's steps (e.g., first sensor information (1401) and second sensor information (1402)) may be information acquired at the same time.

[0151] For example, the electronic device (200) can recognize the user's activity (1403) based on the first sensor information (1401) and the second sensor information (1402) at a first time point obtained through the sensor module (210). Additionally, the electronic device (200) can store (1404) the first sensor information (1401) and the second sensor information (1402) at a first time point inside the electronic device (200) (e.g., memory (230)) or outside (e.g., a server).

[0152] According to one embodiment, the electronic device (200) may also use the first sensor information (1401) and the second sensor information (1402) of the first time point for recognizing the user's steps. For example, the electronic device (200) may select (1407) one of the stored first sensor information (1401) and the second sensor information (1402) of the first time point based on the activity recognition result (1405) and use it for step recognition.

[0153] According to an embodiment, when a recognition result related to walking activity is confirmed, the electronic device (200) may recognize (1409) the user's steps (e.g., number of steps) based on the first sensor information (1401) stored at the first time point and then provide the recognition result (1411). Additionally, when a recognition result related to running activity is confirmed, the electronic device (200) may recognize (1409) the user's steps based on the second sensor information (1402) stored at the first time point and then provide the recognition result (1411).

[0154] However, this is merely one example, and the embodiments disclosed in this document are not limited thereto. For example, the inertial information used to recognize the user's activity and the user's steps may be information acquired at different points in time. Specific embodiments related thereto will be described in detail through FIG. 14b below.

[0155]

[0156] FIG. 14b is a drawing for explaining the step recognition operation of an electronic device according to one embodiment.

[0157] Referring to FIG. 14b, an electronic device (200) (e.g., processor (240)) can recognize a user's activity (1413) based on first sensor information (1411) (e.g., 3-axis angular velocity information) and second sensor information (1412) (e.g., 3-axis acceleration information) at a first time point obtained through a sensor module (210) (e.g., second sensor (212)).

[0158] According to one embodiment, the electronic device (200) can select one of the first sensor information (1411-1) and the second sensor information (1412-2) at a second time point based on the activity recognition result (1415) and use it for step recognition.

[0159] According to an embodiment, when a recognition result related to walking activity is confirmed, the electronic device (200) may recognize the user's steps (e.g., number of steps) based on the first sensor information (1411-1) at a second time point obtained through the sensor module (210) and then provide the recognition result (1421). Additionally, when a recognition result related to running activity is confirmed, the electronic device (200) may recognize the user's steps (1419) based on the second sensor information (1411-2) at a second time point obtained through the sensor module (210) and then provide the recognition result (1421). For example, the second time point may be after the user's activity has been recognized.

[0160]

[0161] An electronic device (200) according to one embodiment may include at least one processor (240), at least one sensor (210), and a memory (230) that is operatively connected to the at least one processor (240) and the at least one sensor (210) and stores at least one instruction.

[0162] According to one embodiment, when the at least one instruction is executed individually or collectively by the at least one processor (240), the electronic device (200) may be configured to: acquire first sensor information (810) related to angular velocity and second sensor information (820) related to acceleration through the at least one sensor (210), detect user activity including walking activity or running activity based on the first sensor information and the second sensor information, measure the number of steps of the user based on the first sensor information while the user's walking activity is detected, and measure the number of steps of the user based on the second sensor information while the user's running activity is detected.

[0163] According to one embodiment, the user's activity may further include a stopping activity. According to one embodiment, when the at least one instruction is executed individually or collectively by the at least one processor (240), the electronic device (200) may be configured to: stop the operation of measuring the number of steps of the user while the user's stopping activity is detected.

[0164] According to one embodiment, when the at least one instruction is executed individually or collectively by the at least one processor (240), the electronic device (200) may be configured to stop the operation of measuring the number of steps of the user until the walking activity or the running activity is detected.

[0165] According to one embodiment, when the at least one instruction is executed individually or collectively by the at least one processor (240), the electronic device (200) may be configured to: measure the number of steps of the user based on the first sensor information when a first type of arm movement is detected while the user's walking activity is detected. For example, the first type of arm movement may include a movement of the arm swinging like a pendulum during the walking activity.

[0166] According to one embodiment, when the at least one instruction is executed individually or collectively by the at least one processor (240), the electronic device (200) may be configured to: convert the first sensor information into the frequency domain when a second type of arm movement different from the first type is detected while the user’s walking activity is detected, measure the number of steps of the user based on information corresponding to a designated reference frequency among the first sensor information in the converted frequency domain, and exclude information among the first sensor information in the converted frequency domain that does not correspond to the designated reference frequency from the measurement of the number of steps of the user.

[0167] According to one embodiment, when the at least one instruction is executed individually or collectively by the at least one processor (240), the electronic device (200) may be configured to: store the first sensor information and the second sensor information at a first time point used to detect the user's activity, measure the number of steps of the user based on the stored first sensor information while the user's walking activity is detected, and measure the number of steps of the user based on the stored second sensor information while the user's running activity is detected.

[0168] According to one embodiment, when the at least one instruction is executed individually or collectively by the at least one processor (240), the electronic device (200) may be configured to: detect the activity of the user based on the first sensor information and the second sensor information at a first time point obtained through the at least one sensor (210); measure the number of steps of the user based on the first sensor information at a second time point obtained through the at least one sensor (210) while the user’s walking activity is detected; and measure the number of steps of the user based on the second sensor information at a second time point obtained through the at least one sensor (210) while the user’s running activity is detected.

[0169] According to one embodiment, when the at least one instruction is executed individually or collectively by the at least one processor (240), the electronic device (200) may be configured to input the first sensor information and the second sensor information into an artificial intelligence model (232) stored in the memory (230), and to detect the user's activity based on the output of the artificial intelligence model (232).

[0170] According to one embodiment, the artificial intelligence model (232) may include a convolutional layer (1020) configured to generate a feature map from the first sensor information and the second sensor information. For example, the convolutional layer may include at least one general computational layer using a number of filters equal to the number of feature maps to be generated and at least one lightweight computational layer using a number of filters less than the number of feature maps to be generated.

[0171] According to one embodiment, the electronic device (200) may include a watch-shaped wearable device.

[0172]

[0173] FIG. 15 is a flowchart illustrating the operation of an electronic device according to various embodiments. In addition, each operation in the following embodiments may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Furthermore, at least one of the aforementioned operations may be omitted depending on the embodiment.

[0174] Referring to FIG. 15, an electronic device (200) (e.g., processor (240)) according to one embodiment can acquire sensor information in operation 1510. For example, the electronic device (200) can acquire first sensor information (1401) (e.g., 3-axis angular velocity information) and second sensor information (1402) (e.g., 3-axis acceleration information) through a sensor module (210) (e.g., second sensor (212)).

[0175] According to one embodiment, an electronic device (200) (e.g., processor (240)) can detect user activity based on sensor information in operation 1520. For example, user activity may include at least walking activity and running activity. According to an embodiment, an activity recognition model (232) configured to recognize (or detect) user activity based on sensor information may be stored inside the electronic device (200) (e.g., memory (230)) and / or outside (e.g., server). In this case, the electronic device (200) can detect user activity using sensor information and the activity recognition model (232).

[0176] According to one embodiment, an electronic device (200) (e.g., processor (240)) may, in operation 1530, select sensor information to be used for step recognition based on the user's activity. For example, when the user's first activity is detected (or identified), the electronic device (200) may select one of acceleration information or angular velocity information that is not affected by the first activity and use it for step recognition. Additionally, when the user's second activity is detected (or identified), the electronic device (200) may select the other of acceleration information or angular velocity information that is not affected by the second activity and use it for step recognition. According to an embodiment, one of the first activity or the second activity may include a walking activity, and the other may include a running activity.

[0177] According to one embodiment, an electronic device (200) (e.g., processor (240)) can recognize a user's step using selected sensor information in operation 1540. For example, the electronic device (200) can utilize information among the selected sensor information that satisfies specified conditions for step recognition.

[0178]

[0179] FIG. 16 is a flowchart illustrating the step recognition operation of an electronic device according to various embodiments. The operations of FIG. 16 described below may represent various embodiments of the operation 1530 of FIG. 15.

[0180] Referring to FIG. 16, an electronic device (200) (e.g., processor (240)) according to one embodiment can determine a detection result (or identification result) of a user's activity in operation 1610. For example, the electronic device (200) can determine whether walking activity is detected or running activity is detected.

[0181] According to one embodiment, the electronic device (200) (e.g., processor (240)) can determine whether arm movement is detected in operation 1620 when a detection result related to walking activity is confirmed. The arm movement may be a second type of arm movement that may cause a decrease in step recognition performance. For example, the electronic device (200) can determine whether a second type of arm movement occurs during the user's walking activity.

[0182] According to one embodiment, an electronic device (200) (e.g., processor (240)) can utilize angular velocity information that is not significantly affected by walking activity (or angular velocity information that better reflects walking activity) for step recognition in operation 1630 when a second type of arm movement does not occur during the user's walking activity.

[0183] According to one embodiment, an electronic device (200) (e.g., processor (240)) can utilize a step frequency for step recognition in operation 1640 when a second type of arm movement occurs during a user's walking activity. For example, the electronic device (200) can convert angular velocity magnitude information in the time domain into the frequency domain and utilize the converted information in the frequency domain and a pre-specified reference step frequency for step recognition.

[0184] According to one embodiment, if a detection result not related to walking activity is confirmed in operation 1610, the electronic device (200) (e.g., processor (240)) can determine in operation 1650 whether a detection result related to the user's running activity (or running motion) is confirmed or whether a stopping activity is detected.

[0185] According to one embodiment, when a detection result related to running activity is confirmed, an electronic device (200) (e.g., processor (240)) can utilize acceleration information that is not significantly affected by running activity (or acceleration information that better reflects running activity) for step recognition in operation 1660.

[0186] According to one embodiment, the electronic device (200) (e.g., processor (240)) may stop the step recognition operation if walking activity and running activity are not detected (e.g., if a detection result related to a stationary activity is confirmed). For example, the electronic device (200) may stop the step recognition operation until walking activity or running activity is detected to prevent a stationary state from being misidentified as a step.

[0187] In the above-described embodiment, a step recognition operation of an electronic device (200) according to one embodiment has been described. However, this is merely one embodiment, and the embodiments disclosed in this document are not limited thereto. For example, at least some of the above-described step recognition operations may be performed by another electronic device (e.g., a smartphone). For example, first sensor information (1401) and second sensor information (1402) may be acquired by the electronic device (200). Additionally, at least some of the user activity detection operation and step recognition operation may be performed by another electronic device.

[0188]

[0189] FIG. 17 is a block diagram of an electronic device (1701) in a network environment (1700) according to various embodiments.

[0190] Referring to FIG. 17, in a network environment (1700), an electronic device (1701) may communicate with an electronic device (1702) through a first network (1798) (e.g., a short-range wireless communication network) or with at least one of an electronic device (1704) or a server (1708) through a second network (1799) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (1701) may communicate with the electronic device (1704) through a server (1708). According to one embodiment, the electronic device (1701) may include a processor (1720), memory (1730), input module (1750), sound output module (1755), display module (1760), audio module (1770), sensor module (1776), interface (1777), connection terminal (1778), haptic module (1779), camera module (1780), power management module (1788), battery (1789), communication module (1790), subscriber identification module (1796), or antenna module (1797). In some embodiments, at least one of these components (e.g., connection terminal (1778)) may be omitted from the electronic device (1701), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (1776), camera module (1780), or antenna module (1797)) may be integrated into a single component (e.g., display module (1760)).

[0191] The processor (1720) can, for example, execute software (e.g., program (1740)) to control at least one other component (e.g., hardware or software component) of the electronic device (1701) connected to the processor (1720) and perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (1720) can store commands or data received from other components (e.g., sensor module (1776) or communication module (1790)) in volatile memory (1732), process the commands or data stored in volatile memory (1732), and store the resulting data in non-volatile memory (1734). According to one embodiment, the processor (1720) may include a main processor (1721) (e.g., a central processing unit or an application processor) or an auxiliary processor (1723) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (1701) includes a main processor (1721) and an auxiliary processor (1723), the auxiliary processor (1723) may be configured to use less power than the main processor (1721) or to be specialized for a specified function. The auxiliary processor (1723) may be implemented separately from the main processor (1721) or as part thereof.

[0192] The auxiliary processor (1723) may control at least some of the functions or states associated with at least one component of the electronic device (1701) (e.g., display module (1760), sensor module (1776), or communication module (1790)) on behalf of the main processor (1721) while the main processor (1721) is in an inactive (e.g., sleep) state, or together with the main processor (1721) while the main processor (1721) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (1723) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (1780) or communication module (1790)). According to one embodiment, the auxiliary processor (1723) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (1701) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (1708)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0193] The memory (1730) can store various data used by at least one component of the electronic device (1701) (e.g., processor (1720) or sensor module (1776)). The data may include, for example, software (e.g., program (1740)) and input or output data for related commands. The memory (1730) may include volatile memory (1732) or non-volatile memory (1734).

[0194] The program (1740) may be stored as software in memory (1730) and may include, for example, an operating system (1742), middleware (1744), or an application (1746).

[0195] The input module (1750) can receive commands or data to be used for a component of the electronic device (1701) (e.g., processor (1720)) from outside the electronic device (1701) (e.g., user). The input module (1750) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0196] The sound output module (1755) can output a sound signal to the outside of the electronic device (1701). The sound output module (1755) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0197] The display module (1760) can visually provide information to an external (e.g., user) of the electronic device (1701). The display module (1760) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (1760) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0198] The audio module (1770) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (1770) can acquire sound through the input module (1750) or output sound through the sound output module (1755) or an external electronic device (e.g., electronic device (1702), speaker or headphones) connected directly or wirelessly to the electronic device (1701).

[0199] The sensor module (1776) can detect the operating state of the electronic device (1701) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (1776) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0200] The interface (1777) may support one or more specified protocols that can be used for the electronic device (1701) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (1702)). According to one embodiment, the interface (1777) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0201] The connection terminal (1778) may include a connector through which the electronic device (1701) can be physically connected to an external electronic device (e.g., electronic device (1702)). According to one embodiment, the connection terminal (1778) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0202] The haptic module (1779) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (1779) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0203] The camera module (1780) can capture still images and video. According to one embodiment, the camera module (1780) may include one or more lenses, image sensors, image signal processors, or flashes.

[0204] The power management module (1788) can manage power supplied to the electronic device (1701). According to one embodiment, the power management module (1788) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0205] The battery (1789) can supply power to at least one component of the electronic device (1701). According to one embodiment, the battery (1789) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0206] The communication module (1790) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (1701) and an external electronic device (e.g., electronic device (1702), electronic device (1704), or server (1708)), and the performance of communication through the established communication channel. The communication module (1790) may include one or more communication processors that operate independently of the processor (1720) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1790) may include a wireless communication module (1792) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (1794) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (1704) via a first network (1798) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (1799) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1792) can identify or authenticate the electronic device (1701) within a communication network such as the first network (1798) or the second network (1799) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (1796).

[0207] The wireless communication module (1792) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (1792) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (1792) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (1792) can support various requirements specified in the electronic device (1701), external electronic device (e.g., electronic device (1704)), or network system (e.g., second network (1799)). According to one embodiment, the wireless communication module (1792) can support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0208] An antenna module (1797) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (1797) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (1797) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (1798) or a second network (1799), may be selected from the plurality of antennas, for example, by a communication module (1790). A signal or power may be transmitted or received between the communication module (1790) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (1797).

[0209] According to various embodiments, the antenna module (1797) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0210] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0211] According to one embodiment, commands or data may be transmitted or received between the electronic device (1701) and an external electronic device (1704) through a server (1708) connected to a second network (1799). Each of the external electronic devices (1702, or 104) may be the same or a different type of device as the electronic device (1701). According to one embodiment, all or part of the operations performed on the electronic device (1701) may be performed on one or more of the external electronic devices (1702, 104, or 108). For example, if the electronic device (1701) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (1701) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (1701). The electronic device (1701) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (1701) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (1704) may include an Internet of Things (IoT) device. The server (1708) may be an intelligent server using machine learning and / or neural networks.According to one embodiment, an external electronic device (1704) or server (1708) may be included within the second network (1799). The electronic device (1701) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0212] The electronic device (1701) according to the various embodiments disclosed in this document may be of various forms. The electronic device (1701) may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. The electronic device (1701) according to the embodiments of this document is not limited to the aforementioned devices.

[0213]

[0214] A method of operation of an electronic device (200) according to one embodiment may include: acquiring first sensor information (810) related to angular velocity and second sensor information (820) related to acceleration through at least one sensor; detecting a user’s activity, including walking or running activity, based on the first sensor information and the second sensor information; measuring the number of steps of the user based on the first sensor information while the user’s walking activity is detected; and measuring the number of steps of the user based on the second sensor information while the user’s running activity is detected.

[0215] According to one embodiment, the user's activity may further include a stopping activity. According to one embodiment, the method of operation of the electronic device (200) may include an operation of stopping the operation of measuring the number of steps of the user while the user's stopping activity is detected.

[0216] According to one embodiment, the method of operation of the electronic device (200) may include the operation of stopping the operation of measuring the number of steps of the user until the walking activity or the running activity is detected.

[0217] According to one embodiment, the method of operation of the electronic device (200) may include the operation of measuring the number of steps of the user based on the first sensor information when a first type of arm movement is detected while the user's walking activity is detected. For example, the first type of arm movement may include a movement of the arm swinging like a pendulum during the walking activity.

[0218] According to one embodiment, the method of operation of the electronic device (200) may include, when a second type of arm movement different from the first type is detected while the user's walking activity is detected, converting the first sensor information into the frequency domain, measuring the number of steps of the user based on information corresponding to a designated reference frequency among the first sensor information in the converted frequency domain, and excluding information among the first sensor information in the converted frequency domain that does not correspond to the designated reference frequency from measuring the number of steps of the user.

[0219] According to one embodiment, the method of operation of the electronic device (200) may include the operation of storing the first sensor information and the second sensor information at a first time point used to detect the user's activity, the operation of measuring the number of steps of the user based on the stored first sensor information while the user's walking activity is detected, and the operation of measuring the number of steps of the user based on the stored second sensor information while the user's running activity is detected.

[0220] According to one embodiment, the method of operation of the electronic device (200) may include detecting the activity of the user based on the first sensor information at a first time point and the second sensor information obtained through the at least one sensor, measuring the number of steps of the user based on the first sensor information at a second time point obtained through the at least one sensor while the user's walking activity is detected, and measuring the number of steps of the user based on the second sensor information at a second time point obtained through the at least one sensor while the user's running activity is detected.

[0221] According to one embodiment, the method of operation of the electronic device (200) may include the operation of inputting the first sensor information and the second sensor information into an artificial intelligence model (232) stored in the electronic device, and the operation of detecting the user's activity based on the output of the artificial intelligence model.

[0222] According to one embodiment, the artificial intelligence model (232) may include a convolutional layer configured to generate a feature map from the first sensor information and the second sensor information. For example, the convolutional layer may include at least one general computational layer using a number of filters equal to the number of feature maps to be generated and at least one lightweight computational layer using a number of filters less than the number of feature maps to be generated.

[0223] A computer-readable storage medium according to one embodiment may store at least one instruction that, when executed by a processor (240) of an electronic device (200), causes the electronic device (200) to: acquire first sensor information (810) related to angular velocity and second sensor information (820) related to acceleration through at least one sensor (210), detect a user’s activity including walking or running activity based on the first sensor information and the second sensor information, measure the number of steps of the user based on the first sensor information while the user’s walking activity is detected, and measure the number of steps of the user based on the second sensor information while the user’s running activity is detected.

[0224] A step recognition system according to one embodiment may include a first electronic device and a second electronic device. For example, the first electronic device includes a first processor (240), a sensor (210), and a first memory (230) which is operatively connected to the first processor and the sensor and stores at least one first instruction. When the at least one first instruction is executed individually or collectively by the at least one first processor, the first electronic device may be configured to: acquire first sensor information (810) related to angular velocity and second sensor information (820) related to acceleration through the sensor and provide them to the second electronic device. For example, the second electronic device may include a second processor and a second memory operatively connected to the second processor and storing at least one second instruction, and when the at least one second instruction is executed individually or collectively by the at least one second processor, the second electronic device may be configured to: detect a user’s activity including walking or running activity based on the first sensor information and the second sensor information, measure the number of steps of the user based on the first sensor information while the user’s walking activity is detected, and measure the number of steps of the user based on the second sensor information while the user’s running activity is detected.

Claims

1. In an electronic device (200), At least one processor (240); Multiple sensors (210); and It includes a memory (230) that is operatively connected to the above at least one processor and the above plurality of sensors and stores at least one instruction, and When the above at least one instruction is executed individually or collectively by the above at least one processor, the electronic device: Through the plurality of sensors above, first sensor information (810) related to angular velocity and second sensor information (820) related to acceleration are obtained, and Based on the first sensor information and the second sensor information, detecting a user's activity including walking or running activity, and While the walking activity of the user is detected, the number of steps of the user is measured based on the first sensor information, and An electronic device configured to measure the number of steps of the user based on the second sensor information while the user's running activity is detected.

2. In Paragraph 1, The above user's activity further includes a stop activity, When the above at least one instruction is executed individually or collectively by the above at least one processor, the electronic device: An electronic device configured to stop the operation of measuring the number of steps of the user while the user's resting activity is detected.

3. In Paragraph 2, When the above at least one instruction is executed individually or collectively by the above at least one processor, the electronic device: An electronic device configured to stop the operation of measuring the number of steps of the user until the walking activity or the running activity is detected.

4. In Paragraph 1, When the above at least one instruction is executed individually or collectively by the above at least one processor, the electronic device: When a first type of arm movement is detected while the walking activity of the user is detected, the number of steps of the user is set to be measured based on the first sensor information. The first type of arm movement described above is an electronic device that includes a swinging arm motion similar to a pendulum motion during the walking activity.

5. In Paragraph 4, When the above at least one instruction is executed individually or collectively by the above at least one processor, the electronic device: When a second type of arm movement different from the first type is detected while the walking activity of the user is detected, the first sensor information is converted into the frequency domain, and The number of steps of the user is measured based on information corresponding to a designated reference frequency among the first sensor information in the converted frequency domain, and An electronic device configured to exclude information among the first sensor information in the converted frequency range that does not correspond to the specified reference frequency from the user's step count measurement.

6. In Paragraph 1, When the above at least one instruction is executed individually or collectively by the above at least one processor, the electronic device: Storing the first sensor information and the second sensor information at a first time point used for detecting the activity of the above user, While the walking activity of the user is detected, the number of steps of the user is measured based on the stored first sensor information, and An electronic device configured to measure the number of steps of the user based on the stored second sensor information while the user's running activity is detected.

7. In Paragraph 1, When the above at least one instruction is executed individually or collectively by the above at least one processor, the electronic device: Detecting the user's activity based on the first sensor information and the second sensor information at a first time point obtained through the plurality of sensors, and While the walking activity of the user is detected, the user's steps are recognized based on the first sensor information at a second time point obtained through the plurality of sensors, and An electronic device configured to recognize the user's steps based on the second sensor information at the second time point obtained through the plurality of sensors while the user's running activity is detected.

8. In any one of paragraphs 1 through 7, When the above at least one instruction is executed individually or collectively by the above at least one processor, the electronic device: The first sensor information and the second sensor information are input into the artificial intelligence model (232) stored in the memory, and An electronic device configured to detect the activity of the user based on the output of the artificial intelligence model.

9. In Paragraph 8, The artificial intelligence model includes a convolutional layer (1020) configured to generate a feature map from the first sensor information and the second sensor information, and The above-mentioned synthetic layer is, At least one general operation layer (1021) using the same number of filters as the number of feature maps to be generated; and An electronic device comprising at least one lightweight computation layer (1023) using fewer filters than the number of feature maps to be generated.

10. In Paragraph 1, The above electronic device is an electronic device including a watch-shaped wearable device.

11. In the method of operating the electronic device (200), The operation of obtaining first sensor information (810) related to angular velocity and second sensor information (820) related to acceleration through a sensor; An operation to detect a user's activity, including walking or running activity, based on the first sensor information and the second sensor information; An operation to measure the number of steps of the user based on the first sensor information while the walking activity of the user is detected; and A method comprising the operation of measuring the number of steps of the user based on the second sensor information while the user's running activity is detected.

12. In Paragraph 11, The above user's activity further includes a stop activity, A method comprising an action of stopping the action of measuring the number of steps of the user while the user’s resting activity is detected.

13. In Paragraph 12, A method for stopping the operation of measuring the number of steps of the user until the walking activity or the running activity is detected.

14. In Paragraph 11, When a first type of arm movement is detected while the walking activity of the above user is detected, the number of steps of the above user is measured based on the first sensor information, and The above-mentioned first type of arm movement includes a method of swinging the arm like a pendulum during the above-mentioned walking activity.

15. In Paragraph 14, When a second type of arm movement different from the first type is detected while the walking activity of the user is detected, the operation of converting the first sensor information into the frequency domain; An operation to measure the number of steps of the user based on information corresponding to a designated reference frequency among the first sensor information in the converted frequency domain; and A method comprising the operation of excluding from the measurement of the user's step count information information among the first sensor information in the converted frequency domain that does not correspond to the specified reference frequency.

Images