Sample acquisition method, device, terminal and storage medium
Through terminal sensor acquisition and automatic labeling, the problem of low efficiency in obtaining machine learning training samples is solved, and the target samples are efficiently acquired and cost is reduced.
Patent Information
- Application Number
- CN202110547708.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-19
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-05-19
AI Technical Summary
In the field of machine learning, obtaining high-quality training samples requires a large amount of manual annotation, which is inefficient and costly.
The sensors in the terminal collect original sample data with a first time length, and automatically determine the start sampling frame and the end sampling frame, intercept the corresponding sampling frame as the target sample, and use it to train a machine learning model that recognizes the flip action of the terminal.
The steps of manual annotation are reduced, the acquisition efficiency of target samples is improved, and the cost is reduced.
Smart Images

Figure CN113283493B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a sample acquisition method, device, terminal and storage medium. Background Art
[0002] In the field of machine learning technology, in order to obtain a higher performance machine learning model, a large number of training samples are required to train the initial machine learning model.
[0003] In the related art, training samples are data labeled with output results. After the data is collected, it is usually necessary to manually label the data to generate training samples. Summary of the invention
[0004] The embodiment of the present application provides a method, device, terminal and storage medium for obtaining a sample. The technical solution is as follows:
[0005] According to one aspect of the present application, a method for obtaining a sample is provided, the method comprising:
[0006] Collecting original sample data having a first duration, wherein the original sample data is used to indicate a change in posture of the terminal within the first duration;
[0007] Determine a start sampling frame and an end sampling frame in the original sample data, wherein the sampling frame between the start sampling frame and the end sampling frame is used to indicate a flipping action process performed by the terminal;
[0008] The sampling frames between the start sampling frame and the end sampling frame are intercepted as target samples, and the target samples are used to train a machine learning model, and the machine learning model is used to identify the flipping action generated by the terminal.
[0009] According to another aspect of the present application, a sample acquisition device is provided, the device comprising:
[0010] A data collection module, used for collecting raw sample data of a first duration, wherein the raw sample data is used for indicating a posture change of the terminal within the first duration;
[0011] A frame determination module, used to determine a start sampling frame and an end sampling frame in the original sample data, wherein the sampling frame between the start sampling frame and the end sampling frame is used to indicate a flipping action process performed by the terminal;
[0012] A sample acquisition module is used to intercept the sampling frames between the start sampling frame and the end sampling frame as target samples, and the target samples are used to train a machine learning model, and the machine learning model is used to identify the flipping action generated by the terminal.
[0013] According to another aspect of the present application, a terminal is provided, which includes a processor and a memory, wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement a sample acquisition method as provided in various aspects of the present application.
[0014] According to another aspect of the present application, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement a sample acquisition method as provided in various aspects of the present application.
[0015] According to one aspect of the present application, a computer program product is provided, the computer program product comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned sample acquisition aspect or the method provided in various optional implementations of the sample acquisition aspect.
[0016] The beneficial effects brought by the technical solution provided by the embodiments of the present application may include:
[0017] The present application collects raw sample data of a first duration through a sensor in a terminal. After obtaining the raw sample data, a start sampling frame and an end sampling frame are determined from the raw sample, wherein the sampling frame between the start sampling frame and the end sampling frame is used to indicate a flipping action process performed by the terminal, and the sampling frame between the start sampling frame and the end sampling frame is intercepted as a target sample to train a machine learning model that can identify the flipping action generated by the terminal. Since the present application can automatically determine the start sampling frame and the end sampling frame in the raw sample data when collecting the target sample, the step of manually confirming the above frames is reduced, thereby improving the efficiency of obtaining the target sample. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly introduce the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 It is a schematic diagram of a sample collection process involved in an embodiment of the present application;
[0020] Figure 2 is a structural block diagram of a terminal provided by an exemplary embodiment of the present application;
[0021] Figure 3 is a flow chart of a method for obtaining a sample provided by an exemplary embodiment of the present application;
[0022] Figure 4 is a flow chart of a method for obtaining a sample provided by another exemplary embodiment of the present application;
[0023] Figure 5 is based on Figure 4 A schematic diagram of a user interface of a sample collection application provided by the illustrated embodiment;
[0024] Figure 6 is based on Figure 5 A schematic diagram of a terminal posture provided by the illustrated embodiment;
[0025] Figure 7 is based on Figure 4 A schematic diagram of raw sample data collected by a terminal provided by the illustrated embodiment;
[0026] Figure 8 is based on Figure 4 A schematic diagram of another type of original sample data collected by a terminal provided in the illustrated embodiment;
[0027] Fig. 9 is based on Figure 4 A schematic diagram of another type of original sample data collected by a terminal provided in the illustrated embodiment;
[0028] Fig.10 is based on Figure 4 A schematic diagram of another type of original sample data collected by a terminal provided in the illustrated embodiment;
[0029] Fig.11 is based on Figure 4 A schematic diagram of another type of original sample data collected by a terminal provided in the illustrated embodiment;
[0030] Fig.12 is based on Figure 4 A schematic diagram of another type of original sample data collected by a terminal provided in the illustrated embodiment;
[0031] Fig.13 is based on Figure 4 A schematic diagram of another type of original sample data collected by a terminal provided in the illustrated embodiment;
[0032] Fig.14 It is a structural block diagram of a sample acquisition device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0033] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0034] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.
[0035] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.
[0036] In the field of machine learning, in order to improve the recognition or judgment ability of machine learning models, technicians usually need to provide a large number of samples with corresponding results marked in the samples. In the process of collecting samples, technicians usually need to manually collect and manually mark the results.
[0037] In a possible sampling scenario, samples may be used to identify terminal actions. In this scenario, the terminal may generate different actions as the user holds it, such as flipping or shaking. To facilitate the training of the machine learning model, the sample can be set to a specified length, for example, 1 second, 1.5 seconds, or 2 seconds. In this sampling scenario, technicians can treat the sensor data within the time length as a set, labeled with labels such as "flipping action", "shaking action", or "negative sample". Among them, "negative sample" indicates that the sample is a negative sample for training the machine learning model, and is also used to improve the training effect of the machine learning model.
[0038] Taking the duration of a sample as 1 second, the sampling frame rate as 60fps (frames per second), and the total acquisition time as 5 seconds as an example, a sample acquisition process is described. In the related art, a technician can hold a terminal, click the start acquisition button, and the handheld terminal performs a flip action. At the same time, the process is captured by a camera. When the process is completed, a data set of 300 frames with a length of 5 seconds and an image of the terminal's flipping action within the 5 seconds are currently obtained. Subsequently, the designer will determine the starting data frame of the flipping action from the 300 frames based on the image, and capture the data frames within 1 second after the frame as a data set. The data set is labeled "flip action" to achieve the acquisition of a sample.
[0039] See also Figure 1 , Figure 1 Schematic diagram of a sample collection process involved in an embodiment of the present application. Figure 1 In the process, the technician turns on the camera 1A to start recording. Then, the technician holds the terminal 1B and clicks the start acquisition button on the user interface, and performs a flipping action within the shooting range of the camera 1A. After the start acquisition button in the terminal 1B is clicked for 5 seconds, the acquisition of sensor data from the sensor of the terminal 1B is automatically stopped. The sensor can be any one of a gravity sensor, a gyroscope or a magnetometer. The camera 1A obtains and saves the video file 110, and the terminal 1B obtains and saves the original sample data 120.
[0040] For illustration, please refer to Table 1, which shows the contents that may be included in a sampling frame. In Table 1, a sampling frame includes the x-axis gravity acceleration, the y-axis gravity acceleration and the z-axis gravity acceleration.
[0041] Table 1 takes 3 sampling frames as an example for explanation.
[0042]
[0043] Table 1
[0044] exist Figure 1 Each of the 300 sampling frames of the original sample data 120 shown can be a set of data in the sampling frames shown in Table 1. The technician will determine the specific moment when the terminal 1B starts the flipping action by playing the video file 110, and determine the moment as the start sampling frame. Since the sampling frame rate is 60fps and the required sample duration is 1 second, the technician will intercept the 60 frames including the start sampling frame as a sample and mark the sample with a label of "flip action".
[0045] As can be seen from the above examples, current sample collection technology requires technicians to do a lot of work to obtain data that can reflect the actual action process.
[0046] In order to simplify the steps of collecting samples and improve the efficiency of sample collection, the present application provides the following sample acquisition method.
[0047] In order to facilitate the understanding of the scheme shown in the embodiments of the present application, several nouns appearing in the embodiments of the present application are introduced below.
[0048] Original sample data: data containing noise collected by the terminal, which is used to indicate the posture change of the terminal within the first time length. Among them, the original sample data may include two parts: noise and target samples. Optionally, the noise may include at least one of head noise and tail noise. In one possible way, the original sample data includes two parts: head noise and target samples. In another possible way, the original sample data includes two parts: tail noise and target samples. In another possible way, the original sample data includes three parts: head noise, target sample and tail noise.
[0049] Optionally, the first duration is a preset constant value, which can be a constant set by the terminal according to the configuration file, or a value input by the user of the terminal through the user interface (UI) of the terminal. The value of the first duration can be a time length of 3 seconds, 4 seconds, 5 seconds or 6 seconds. In a possible setting, the value of the first duration is greater than the duration of the target sample. For example, if the length of the sample that the terminal needs to collect is 2 seconds, the first duration will be greater than 2 seconds.
[0050] Optionally, the posture change includes the process from the initial posture to the end posture. The original sample data includes both the data corresponding to the initial posture and the data corresponding to the end posture. The original sample data records the corresponding data in the process from the initial sample data to the end sample data. In the actual implementation process, the terminal collects corresponding data through a sensor, and the data can be at least one of the gravitational acceleration or the angular acceleration of rotation. Since the sensor data can reflect the posture of the terminal, the terminal indicates the posture change of the terminal by collecting sensor data. The original data indicates the posture change of the terminal within the first time length through the sensor data distributed over the first time length.
[0051] It should be noted that due to the presence of noise in the original sample data, the original sample data cannot be directly used to train the machine learning model. The original sample data needs to be trimmed to obtain the target sample. Those skilled in the art will use the target sample to train the machine learning model. Among them, according to the annotation and data content of the target sample, as well as the architecture of the machine learning model, the idea of this application can be applied to various scenarios for obtaining samples.
[0052] Start sampling frame: It is the first frame of the target sample in the original sample data. From the start sampling frame onwards, it is the data of the target sample. The target sample ends at the end sampling frame.
[0053] End sampling frame: It is the last frame of the target sample in the original sample data. From the end sampling frame to the start sampling frame, they are all target samples.
[0054] For example, if the target sample includes 300 frames, the first frame is the start sampling frame, the 300th frame is the end sampling frame, and the middle 298 frames are middle sampling frames. It should be noted that the number of frames of the target sample is only for illustrative purposes and does not limit the data length of the target sample in this application.
[0055] Target samples: Sample data used to train the machine learning model. In one possible approach, the data lengths of the target samples are equal.
[0056] For ease of understanding, this application uses an example of a machine learning model being used to identify a flipping action generated by a terminal.
[0057] For example, the sample acquisition method shown in the embodiment of the present application can be applied in a terminal, which has a display screen and a sample acquisition function. The terminal can include a handheld mobile terminal such as a mobile phone, a tablet computer or a smart watch. Alternatively, the terminal can also be an engineering terminal specifically used for sampling, which is not limited in the present application.
[0058] Figure 2 is a structural block diagram of a terminal provided by an exemplary embodiment of the present application, such as Figure 2 As shown, the terminal includes a processor 220 and a memory 240, wherein the memory 240 stores at least one instruction, and the instruction is loaded and executed by the processor 220 to implement the sample acquisition method as described in each method embodiment of the present application.
[0059] In the present application, terminal 200 is an electronic device with the function of acquiring samples. Terminal 200 collects raw sample data of a first duration, and the raw sample data is used to indicate the posture change of the terminal within the first duration; a start sampling frame and an end sampling frame are determined in the raw sample data, and the sampling frames between the start sampling frame and the end sampling frame are used to indicate the process of a flipping action performed by the terminal; the sampling frames between the start sampling frame and the end sampling frame are intercepted as target samples, and the target samples are used to train a machine learning model, and the machine learning model is used to identify the flipping action generated by the terminal.
[0060] The processor 220 may include one or more processing cores. The processor 220 uses various interfaces and lines to connect various parts of the entire terminal 200, and executes various functions and processes data of the terminal 200 by running or executing instructions, programs, code sets or instruction sets stored in the memory 240, and calling data stored in the memory 240. Optionally, the processor 220 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 220 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 220, but may be implemented separately through a chip.
[0061] The memory 240 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 240 includes a non-transitory computer-readable storage medium. The memory 240 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 240 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data involved in the following various method embodiments, etc.
[0062] Please refer to Figure 3 , Figure 3 is a flow chart of a sample acquisition method provided by an exemplary embodiment of the present application. The sample acquisition method can be applied to the above Figure 1 in the terminal shown. Figure 3 In the example, the sample acquisition method includes:
[0063] Step 310: collecting original sample data of a first duration, where the original sample data is used to indicate a posture change of the terminal within the first duration.
[0064] In an embodiment of the present application, a system-native sample collection application (Application, App) may be pre-installed in the terminal, and the sample collection application may be a system-native application or a third-party application. When the terminal is an engineering machine specifically used to collect samples, the sample collection application may be a system-native application. When the terminal is an ordinary mobile device on the market, the sample collection application may be a third-party application. Optionally, the user may execute the sample acquisition method provided in the present application through the sample collection application in the terminal.
[0065] Among them, the sample acquisition application can collect raw sample data with a total duration of the first duration. Schematically, the sample acquisition application can sample at a given sampling frame rate when collecting data. For example, if the given sampling frame rate is 60fps, the sample acquisition application will collect 60 sampling frames within 1 second, and each sampling frame includes a set of sensor data. Schematically, the given sampling frame rate can also be any value that can be used by the sample acquisition application, such as 30fps, 90fps or 120fps, and the embodiments of the present application are not limited to this.
[0066] In one possible manner, if the first duration is 5 seconds, the given sampling frame rate is 60 fps. In this scenario, the original sample data is data including 300 frames.
[0067] Step 320: determine a start sampling frame and an end sampling frame in the original sample data, and the sampling frames between the start sampling frame and the end sampling frame are used to indicate a process of a flipping action performed by the terminal.
[0068] After acquiring the original sample data, the terminal will automatically determine the start sampling frame and the end sampling frame therein. It should be noted that the moment of the start sampling frame will be earlier than the moment of the end sampling frame. The sampling frames between the start sampling frame and the end sampling frame are used to indicate the process of a flipping action performed by the terminal. Since the terminal performed a flipping action within the first time period, there is data corresponding to a period of time in the original sample data, which is used to fully reflect the process of the flipping action. In this step, the terminal will automatically determine the start sampling frame and the end sampling frame of the process that reflects the flipping action.
[0069] In step 330, the sampling frames between the start sampling frame and the end sampling frame are intercepted as target samples. The target samples are used to train the machine learning model, and the machine learning model is used to identify the flipping action generated by the terminal.
[0070] In the embodiment of the present application, after the terminal automatically determines the start sampling frame and the end sampling frame, the terminal uses the two frames as reference points to extract the target sample including the start sampling frame and the end sampling frame from the original sample data. It should be noted that the target sample is used to train the machine learning model, and the trained machine learning model is used to identify the flipping action generated by the terminal.
[0071] Optionally, the number of target samples can be 10 2 or 10 3 order of magnitude.
[0072] Optionally, the target sample can be obtained by repeatedly executing it multiple times on the same terminal, or the same sample collection application can be installed on different terminals to collect the target sample. For example, when 500 target samples are needed to train the machine learning model, this solution can make one terminal execute the sample acquisition method 500 times to obtain 500 target samples. This solution can also install the same sample collection application in 5 terminals, and each sample collection application is executed 100 times to obtain 100 target samples. Finally, the target samples collected by each of the 5 terminals are aggregated to obtain 500 samples for training the machine learning model.
[0073] In summary, the sample acquisition method provided in this embodiment can collect original sample data of a first duration through the sensor in the terminal, and after obtaining the original sample data, determine the start sampling frame and the end sampling frame from the original sample, wherein the sampling frame between the start sampling frame and the end sampling frame is used to indicate a flipping action process performed by the terminal, and the sampling frame between the start sampling frame and the end sampling frame is intercepted as a target sample to train a machine learning model that can identify the flipping action generated by the terminal. Since the present application can automatically determine the start sampling frame and the end sampling frame in the original sample data when collecting the target sample, the step of manually confirming the above frames is reduced, thereby improving the efficiency of obtaining the target sample.
[0074] Based on the solution disclosed in the previous embodiment, the terminal can also determine the start sampling frame and the end sampling frame according to different sensor change conditions, thereby automatically determining the target sample. Please refer to the following embodiment.
[0075] See also Figure 4 , Figure 4 FIG. 1 is a flow chart of a method for obtaining a sample provided by another exemplary embodiment of the present application. The method for obtaining a sample can be applied in the terminal shown above. Figure 4 In the example, the method for obtaining the sample includes:
[0076] Step 411: Receive a second trigger signal, where the second trigger signal is a signal generated when a second control in a target application in the terminal is triggered.
[0077] In this application, the terminal can collect raw sample data through the installed target application. The target application indicated in the embodiment of this application is one of the sample collection applications. Among them, the second control is at least one of the data collection button in the sample collection application, the payment code display button in the payment application, the boarding code display button in the public transportation application, or the health code display button in the health application.
[0078] It should be noted that the solution provided in this application can also realize the collection of target samples through the daily use of users' own mobile phones. In this scenario, the manufacturer can provide a shared improvement plan or other sharing plan, so that users can use the data of their own terminal to improve the machine learning model after desensitization. Since the target application has the original functions, the terminal can set up a system service running in the background in the operating system. The system service can respond accordingly based on the various signals received in the target application.
[0079] Step 412, starting from the time when the second trigger signal is received, collects original sample data of a first duration.
[0080] Illustratively, the target application itself may not have the permission to call the corresponding sensor. The present application can be executed normally without changing the target application. During actual execution, the system service in the operating system receives the second trigger signal and can call the sensor to collect raw sample data of the first duration starting from the moment the second trigger signal is received. In this scenario, the system service has the permission to call the corresponding sensor, and the system service has a higher permission, which is set in advance through system version updates and the like.
[0081] In one possible approach, when the target application can be a sample collection application, see Figure 5 , Figure 5 is based on Figure 4 A schematic diagram of a user interface of a sample collection application provided by the illustrated embodiment. In the user interface 500, there are a first starting state check box 511, a second starting state check box 512, and a negative sample check box 513, as well as a first end state check box 521, a second end state check box 522, a third end state check box 523, and a negative sample check box 524, as well as a negative sample collection button 530, a start detection button 540, and a write data button 550. It should be noted that the check boxes are used to select the starting state and the end state of the terminal, and each check box represents a posture.
[0082] Optionally, the first control can be Figure 5 The check box shown. The terminal receives a first trigger signal, which is a signal generated when the first control of the target application in the terminal is triggered. The terminal can determine the preset posture of the terminal corresponding to the sampling frame according to the first trigger signal, or determine the preset posture corresponding to the end sampling frame. Different first controls correspond to different preset postures. For example, the preset posture corresponding to the first starting state check box 511 is the screen-up posture, the preset posture corresponding to the second starting state check box 512 is the screen-back vertical state, the preset posture corresponding to the first end state check box 521 is the screen-front horizontal state, the preset posture corresponding to the second end state check box 522 is the screen-down state, and the preset posture corresponding to the third end state check box 523 is the screen-front vertical posture.
[0083] See also Figure 6 , Figure 6 is based on Figure 5 The embodiment shown is a schematic diagram of a terminal posture. Figure 6, the user holding the terminal 200 is used as a reference. The screen-upward posture 61 indicates that the screen of the terminal 200 is facing the sky and facing away from the ground. The screen-downward posture 62 indicates that the screen of the terminal 200 is facing the ground and facing away from the sky. The screen-front vertical posture 63 indicates that the screen of the terminal 200 faces away from the user and keeps the short side parallel to the ground. The screen-back vertical posture 64 indicates that the screen of the terminal 200 faces the user and keeps the vertical state. The screen-front horizontal posture 65 indicates that the screen of the terminal 200 faces away from the user and keeps the long side parallel to the ground.
[0084] In the user interface 500, the second control is a start detection button 540. When the start detection button 540 is triggered, a second trigger signal is generated in the sample collection application, and the second trigger signal can be received by the system service. Then, the system service receives the second trigger signal, takes the moment of receiving the second trigger signal as the start moment, and collects raw sample data of the first duration.
[0085] In another possible way, when the target application is a payment application, the second control can be a payment code display button in the payment application. It should be noted that the payment code display button in the payment application can be displayed in the user interface of the payment application, can be displayed in the negative one screen of the terminal, and can also be displayed after the payment application is long pressed. The embodiment of the present application does not limit the display position of the payment code display button. When the payment code display button is triggered, the payment application will generate a second trigger signal. After clicking the payment code display button, the user usually flips the phone and displays the payment code displayed in the phone to the barcode scanner or barcode scanner that needs to scan the code. Therefore, it is feasible and practical for the terminal to collect data starting from the second trigger signal, and it is feasible and practical to use the data that lasts for the first period of time as the original sample data.
[0086] Optionally, when the target application is a payment application, the system service can determine the starting posture of the terminal based on the sensor data provided by the first frame of the original sample data, and determine the ending posture of the terminal based on the last frame of the original sample data. Different sensor data can be matched to corresponding starting postures and ending postures. If the corresponding starting posture or ending posture cannot be matched, the system service will discard the original sample data obtained this time and no longer save the data obtained this time.
[0087] In another possible implementation, when the target application is a boarding code display button in a public transportation application or a health code display button in a health application, the display button can be displayed in the lock screen interface in addition to the location where the payment code display button is displayed, so that users can quickly display the boarding code or health code and pass through quickly. After the boarding code display button or the health code display button is triggered, the system service will receive a second trigger signal, and the moment of receiving the second trigger signal will be the start moment, and the original sample data of the first duration will be collected.
[0088] Step 421 : determining a start sampling frame based on a first change condition of the sensor data, and determining a sampling frame of a second duration after the start sampling frame as an end sampling frame, wherein the first change condition is used to indicate that a change rate of the sensor data is greater than a first threshold.
[0089] Optionally, in response to the absolute value of the slope of the sensor data fitting curve at the first sampling frame being less than the first threshold, and the absolute value of the slope of the sensor data fitting curve at the second sampling frame being greater than or equal to the first threshold, the terminal may determine the first sampling frame in the original sample data as the starting sampling frame, the sensor data fitting curve is a curve obtained by fitting the sensor data in each sampling frame, and the first sampling frame is an adjacent sampling frame whose collection time is earlier than the second sampling frame. In one possible manner, the first sampling frame is an adjacent sampling frame earlier than the second sampling frame.
[0090] It should be noted that in the method for determining the start sampling frame and the end sampling frame, the sensor data may be one type of data or multiple types of data. Among them, the sensor data may be gravity acceleration and / or rotational angular acceleration. After collecting the original sample data, the terminal may fit the sensor data curve according to the sensor data in each sampling frame. It should be noted that the sensor data curve has different slopes at different times. Optionally, the present application regards the change of the slope as a change situation. For example, a preset first threshold is set in the terminal, and the terminal regards the change of the absolute value of the slope from a state less than the first threshold to a state greater than or equal to the first slope as the first change situation. From another perspective, the terminal regards the situation where the slope of the qth sampling frame is less than the first threshold and the absolute value of the slope of the q+1th sampling frame is greater than or equal to the first threshold as the first change situation. When the sensor data collected by the terminal meets the first change situation, the terminal will determine the start sampling frame according to the first change situation, and determine the sampling frame of the second duration after the start sampling frame as the end sampling frame. Among them, the second duration can be a fixed duration value such as 1 second, 1.5 seconds or 2 seconds. The second duration may also have other possible constant values, which are not limited in the embodiments of the present application.
[0091] In a possible practical implementation, taking the sampling frame rate as 120fps and the second duration as 1 second as an example, a method for determining the start sampling frame and the end sampling frame is introduced. If the terminal detects that the slope of the sensor data at the 329th sampling frame is less than the first threshold, and the slope of the sensor data at the 330th sampling frame is greater than the first threshold, then the change of the sensor data at this time meets the first change condition. The terminal determines the 330th sampling frame as the start sampling frame, and the 119th sampling frame after the 330th sampling frame, that is, the 449th sampling frame, as the end sampling frame.
[0092] Step 422 , determining an end sampling frame based on a second change condition of the sensor data, and determining a sampling frame of a second duration before the end sampling frame as a start sampling frame, wherein the second change condition is used to indicate that a change rate of the sensor data is less than a second threshold.
[0093] Optionally, in response to the absolute value of the slope of the sensor data fitting curve at the third sampling frame being less than the second threshold, and the absolute value of the slope of the sensor data fitting curve at the fourth sampling frame being greater than or equal to the second threshold, the terminal may determine the fourth sampling frame in the original sample data as the end sampling frame, the sensor data fitting curve being a curve fitted according to the sensor data in each sampling frame, and the third sampling frame being an adjacent sampling frame whose collection time is earlier than the fourth sampling frame; and determine the sampling frame whose collection time is earlier than the end sampling frame by the second time length as the start sampling frame.
[0094] In the embodiment of the present application, the sensor data may be a specified type of sensor data or multiple types of sensor data, wherein the type of sensor data may be gravity acceleration or angular velocity.
[0095] For example, a preset second threshold is set in the terminal, and the terminal changes the absolute value of the slope from a state greater than the absolute value of the second threshold to a state less than or equal to the second slope as the second change situation. From another perspective, the terminal takes the situation where the absolute value of the slope of the rth sampling frame is greater than the second threshold and the absolute value of the slope of the r+1th sampling frame is less than or equal to the first threshold as the second change situation. The terminal can determine the end sampling frame based on the second change situation, and determine the second duration before the end sampling frame as the start sampling frame. Schematically, the second duration can be a fixed duration value such as 1 second, 1.5 seconds or 2 seconds. The second duration can also have other possible constant values, which are not limited in the embodiments of the present application.
[0096] Step 423, determining a start sampling frame based on a first change in the sensor data, and determining an end sampling frame based on a second change in the sensor data, wherein the first change is used to indicate that a change rate of the sensor data is greater than a first threshold, and the second change is used to indicate that a change rate of the sensor data is less than a second threshold.
[0097] Optionally, first, in response to the absolute value of the slope of the sensor data fitting curve at the first sampling frame being less than the first threshold, and the absolute value of the slope of the sensor data fitting curve at the second sampling frame being greater than or equal to the first threshold, the terminal may determine the first sampling frame in the original sample data as the start sampling frame, the sensor data fitting curve is a curve obtained by fitting the sensor data in each sampling frame, and the first sampling frame is an adjacent sampling frame whose collection time is earlier than the second sampling frame. Secondly, in response to the absolute value of the slope of the sensor data fitting curve at the third sampling frame being less than the second threshold, and the absolute value of the slope of the sensor data fitting curve at the fourth sampling frame being greater than or equal to the second threshold, the terminal may determine the fourth sampling frame in the original sample data as the end sampling frame, and the third sampling frame is an adjacent sampling frame whose collection time is earlier than the fourth sampling frame.
[0098] In an embodiment of the present application, the terminal can determine a start sampling frame according to a first change situation, and determine an end sampling frame according to a second change situation.
[0099] Step 431, traverse the sampling frames in the original sample data in sequence according to the acquisition order, and detect whether the posture of the terminal corresponding to the sampling frame meets the preset posture.
[0100] It should be noted that the present application can also determine the end sampling frame by counting. After determining the end sampling frame, the terminal can determine the sampling frame of the second duration before the end sampling frame as the start sampling frame.
[0101] In one possible manner, the terminal may implement the effect of detecting whether the posture of the terminal corresponding to the sampling frame conforms to the preset posture through steps (a1), (a2) and (a3).
[0102] Step (a1), obtaining the posture type of the preset posture.
[0103] In the present application, the terminal can obtain the posture type of the preset posture in the original sampling frame. It should be noted that the posture type can be one of the vertical posture in front of the screen, the upward posture of the screen, or the horizontal posture in front of the screen. Optionally, the terminal can pre-select the posture type according to the user.
[0104] Step (a2), determining a value range of corresponding sensor data according to the posture type, wherein the sensor data includes at least one of gravitational acceleration or rotational angular acceleration.
[0105] It should be noted that the value intervals of the sensor data corresponding to different posture types are different. In one possible manner, if the posture type is a vertical posture in front of the screen, the value interval of the sensor data corresponding to the vertical posture in front of the screen is value interval A. If the posture type is a screen-upward posture, the value interval of the sensor data corresponding to the screen-upward posture is value interval B. If the posture type is a horizontal posture in front of the screen, the value interval of the sensor data corresponding to the horizontal posture in front of the screen is value interval C.
[0106] The terminal will select the corresponding sensor data value range according to different posture types.
[0107] Optionally, in response to the sensor data including gravity acceleration, it is detected whether a measured value of gravity acceleration along a first direction is equal to a constant value of gravity acceleration, and the first direction includes a normal direction outward along the screen of the terminal.
[0108] In the embodiment of the present application, the terminal may be a cuboid or a quasi-cuboid, wherein the sides of the cuboid with the largest area are respectively the screen at the front and the back panel at the rear.
[0109] Optionally, in response to the sensor data including gravitational acceleration and rotational angular acceleration, it is detected whether the measured value of gravitational acceleration along the first direction is equal to a constant value of gravitational acceleration, and it is detected whether the absolute value of rotational angular acceleration is less than a preset threshold.
[0110] Step (a3), detecting whether the sensor data corresponding to the sampling frame conforms to the value range of the sensor data.
[0111] Optionally, the terminal can detect whether the sensor data corresponding to each sampling frame conforms to the value interval of the sensor data. If the sensor data corresponding to the sampling frame is one data, the data conforms to the value interval of the sensor data. If the sensor data corresponding to the sampling frame is multiple data, each of the multiple data needs to conform to the value interval of the corresponding sensor data. In this scenario, if the number of multiple data is q, the number of corresponding value intervals is also q.
[0112] Step 432: In response to the posture of the terminal corresponding to the sampling frame conforming to the preset posture, a counter is set, and the initial value of the counter is zero.
[0113] Optionally, when the posture of the terminal corresponding to the sampling frame meets the preset posture, the terminal sets a counter, and the initial value of the counter is 0.
[0114] Step 433: In response to the posture of the terminal corresponding to the sampling frame detected after the sampling frame conforming to the preset posture, the value of the counter is increased by one.
[0115] In the embodiment of the present application, after the sampling frame that meets the preset posture, when the terminal detects a sampling frame that meets the same preset posture, the terminal adds 1 to the value of the counter.
[0116] In another possible implementation, the terminal can also reset the counter in response to the posture of the terminal corresponding to the sampling frame detected after the sampling frame not meeting the preset posture; for the sampling frames that have not been detected in the m sampling frames, re-execute the steps of traversing the sampling frames in the original sample data in the order of acquisition, and detecting whether the posture of the terminal corresponding to the sampling frame meets the preset posture.
[0117] It should be noted that when the terminal detects a sampling frame that does not meet the preset posture again in the sampling frame after the sampling frame that meets the preset posture, it means that the posture of the terminal is unstable at this time, so the counter is reset. Subsequently, the terminal will re-execute the sampling frames in the original sample data in the order of acquisition for the sampling frames that have not been detected in the m sampling frames, and detect whether the posture of the terminal corresponding to the sampling frame meets the preset posture.
[0118] Step 434: in response to the value of the counter being equal to n, determining the sampling frame as an end sampling frame.
[0119] In the embodiment of the present application, when the value of the counter is equal to n, it means that there are enough sampling frames to meet the needs and the terminal is in a relatively stable posture. Therefore, the terminal can use the sampling frame when the counter is set as the technical sampling frame of the mobile phone posture.
[0120] Step 435: determine the sampling frame whose collection time is earlier than the second duration of the ending sampling frame as the starting sampling frame.
[0121] On this basis, the terminal can determine the sampling frame that is earlier than the end sampling frame by the second duration as the start sampling frame, based on the end sampling frame.
[0122] Step 440: intercept the sampling frames between the start sampling frame and the end sampling frame as target samples.
[0123] In the embodiment of the present application, the execution process of step 440 is the same as the execution process of step 330, which will not be repeated here.
[0124] In a practical application scenario, a sample acquisition method is introduced, which is applied in an action detection data collection application with data pre-labeling function. This method finds the stable frame of gravity acceleration of each axis of the terminal, and determines the action start frame (that is, the start sampling frame) and the end sampling frame based on the stable frame of gravity acceleration, so as to intercept the target sample from the original sample data.
[0125] Taking the machine learning model used to identify the terminal flipping action as an example, the process of training the target sample of the machine learning model is introduced. For example, the sampling time of the original sample data is 6 seconds and the sampling frame rate is 60fps. In this scenario, when the terminal is flipped upside down, the gravity acceleration (gravity) and rotational angular acceleration (gyro) collected by the sensor are shown in Figure 1. Figure 7 , Figure 7 is based on Figure 4 The illustrated embodiment provides a schematic diagram of raw sample data collected by a terminal. Figure 7 The process shown is from the screen up to the screen down. Figure 7 In the example, the first data subset shows the change of the gravitational acceleration in the three axes in the original sample data. The second data subset shows the change of the rotational angular acceleration in the three axes in the original sample data.
[0126] In the first data subset, the horizontal axis represents the frame number of the sampling frame, the vertical axis represents the value of gravity acceleration, the first curve 711 represents the gravity acceleration along the x-axis, the second curve 712 represents the gravity acceleration along the y-axis, and the third curve 713 represents the gravity acceleration along the z-axis.
[0127] In the second data subset, the horizontal axis represents the number of sampling frames, and the vertical axis represents the value of the rotational angular acceleration. In each data subset, the sensor data of each axis is represented by a sensor data fitting curve to represent the change trend. Curve 721 represents the rotational angular acceleration of the x-axis, curve 722 represents the rotational angular acceleration of the y-axis, and curve 723 represents the rotational angular acceleration of the z-axis.
[0128] exist Figure 7 In the first stage, the 0th frame to the 120th frame indicates the stable state of the terminal screen facing upward. That is, the value of the gravity acceleration along the z-axis is close to 9.8, and the rotational angular acceleration along the x-axis, y-axis, and z-axis are all about 0.
[0129] The second stage: Frames 121 to 200 represent the process of the terminal screen flipping upside down, that is, the gravity acceleration of the three axes and the rotational angular acceleration of the three axes all change significantly.
[0130] The third stage: Frames 201 to 360 indicate that the terminal is in a stable state with the screen facing downward, that is, the value of the gravity acceleration along the z-axis is close to 9.8, and the rotational angular accelerations along the x-axis, y-axis, and z-axis are all about 0.
[0131] In the embodiment of the present application, since the terminal screen is facing downward when the terminal flipping action is finished, the designer can determine the thresholds corresponding to each sensor data when the terminal screen is facing downward by observing multiple groups of flipping action curves, that is, by observing the sensor data fitting curves.
[0132] Indicatively, when the start sampling frame of the pre-marked target sample and the receiving sampling frame are in accordance with the sampling order, that is, according to the early and late sampling time, the terminal traverses each frame in the original sample data. When the terminal confirms that 10 or 15 consecutive frames starting from the i-th frame meet the screen downward threshold, the terminal will determine that the terminal screen is in a downward stable state starting from the i-th frame (that is, the i-th frame is a stable frame).
[0133] Optionally, since an upside-down flipping action usually does not exceed 2 seconds, the starting sampling frame can be set to one second before the i-th frame (i.e. Figure 7 The end sampling frame is set to be j frames after the i-th frame (i.e. Figure 7 Timeline 7B in the figure), where j can be 0 or a specified constant, such as 12, 15 or 18. Therefore, through this method, the terminal can mark the frames between the start sampling frame and the end sampling frame of the upside-down action as our positive sample data, and the others as negative samples.
[0134] In the embodiment of the present application, if it is necessary to collect target samples corresponding to multiple actions, the method is similar. The terminal first determines the thresholds corresponding to the multiple end actions, and then searches for stable frames based on these sets of thresholds. Finally, the terminal determines the start sampling frame and the end sampling frame of the action through the stable frame, and thereby achieves the function of automatically pre-marking from the original sample data, thereby obtaining the target sample.
[0135] In another possible implementation, the application of the embodiment of the present application in a collection work application is used as an example for illustration.
[0136] Schematically, the usage scenarios of this application include presenting a payment code, presenting a boarding code, and presenting a health code. Therefore, the target samples collected by the terminal involve two starting actions and three ending actions in this scenario. Schematically, the target sample has six action combinations. The six action combinations include screen up and screen down, screen up and screen front horizontal, screen up and screen front vertical, screen back vertical and screen down, screen back vertical and screen front horizontal, screen back vertical and screen front vertical). Among them, in the posture of the above-mentioned terminal, the up and down, front and back, horizontal and vertical states are defined with the user as the reference, and are used to describe the screen orientation of the terminal.
[0137] See also Figure 8 , Figure 8 is based on Figure 4The illustrated embodiment provides another schematic diagram of raw sample data collected by a terminal. Figure 8 What is shown is the state where the screen is flipped upward to the front horizontal position of the screen. Figure 8 For the reference numerals and their meanings, see Figure 7 The content shown.
[0138] exist Figure 8 In the scene shown, the terminal screen is in a stable state with the screen facing upward from frame 0 to frame 150. The value of the gravity acceleration in the z-axis direction is close to 9.8, and the values of the rotational angular acceleration in the x-axis, y-axis, and z-axis directions are all approximately equal to 0.
[0139] Frames 150 to 230 are the process of the screen flipping upward to the horizontal position. During the flipping process, the gravitational acceleration along the x-axis, y-axis, and z-axis and the rotational angular velocity along the x-axis, y-axis, and z-axis all change significantly.
[0140] The 230th to 360th frames are the horizontal stable state of the screen. Among them, the value of the gravity acceleration in the z-axis direction is close to -9.8, and the values of the rotational angular acceleration in the x-axis, y-axis and z-axis are all approximately equal to 0.
[0141] See also Fig. 9 , Fig. 9 is based on Figure 4 The illustrated embodiment provides another schematic diagram of raw sample data collected by a terminal. Fig. 9 What is shown is the state where the screen is flipped upward to the front portrait orientation. Fig. 9 For the reference numerals and their meanings, see Figure 7 The content shown.
[0142] exist Fig. 9 In the scene shown, the terminal screen is in a stable state with the screen facing upward from frame 0 to frame 150. The value of the gravity acceleration in the z-axis direction is close to 9.8, and the values of the rotational angular acceleration in the x-axis, y-axis, and z-axis directions are all approximately equal to 0.
[0143] Frames 150 to 240 are the process of the screen flipping upward to the front portrait. During this flipping process, the gravitational acceleration of the three axes, x-axis, y-axis and z-axis, and the rotational angular velocity of the three axes, x-axis, y-axis and z-axis, all change significantly.
[0144] The 240th to 360th frames are the stable state of the screen in the vertical direction. Among them, the value of the gravity acceleration in the y-axis direction is close to 9.8, and the values of the rotational angular acceleration in the three axes of the x-axis, y-axis and z-axis are all approximately equal to 0.
[0145] See also Fig.10 , Fig.10 is based on Figure 4The illustrated embodiment provides another schematic diagram of raw sample data collected by a terminal. Fig.10 The screen is shown as being flipped vertically to face downward. Fig.10 For the reference numerals and their meanings, see Figure 7 The content shown.
[0146] exist Fig.10 In the scene shown, the 0th frame to the 100th frame is a stable state of the terminal screen back vertically, wherein the value of the gravity acceleration in the y-axis direction is greater than 5, the value of the gravity acceleration in the z-axis direction fluctuates around 5, and the gravity acceleration in the x-axis direction fluctuates around 0.
[0147] Frames 100 to 260 are the process of the screen flipping from vertical to downward. During this flipping process, the gravitational acceleration along the x-axis, y-axis, and z-axis and the rotational angular velocity along the x-axis, y-axis, and z-axis all change significantly.
[0148] The 260th to 360th frames are the stable state of the screen facing downward, where the value of the gravity acceleration in the z-axis direction is close to -9.8, and the values of the rotational angular acceleration in the x-axis, y-axis, and z-axis directions are all approximately equal to 0.
[0149] See also Fig.11 , Fig.11 is based on Figure 4 The illustrated embodiment provides another schematic diagram of raw sample data collected by a terminal. Fig.11 What is shown is the state where the back of the screen is flipped from the vertical position to the horizontal position at the front of the screen. Fig.11 For the reference numerals and their meanings, see Figure 7 The content shown.
[0150] exist Fig.11 In the scene shown, the 0th frame to the 100th frame is a stable state of the terminal screen back vertically, wherein the value of the gravity acceleration in the y-axis direction is greater than 5, the value of the gravity acceleration in the z-axis direction fluctuates around 5, and the gravity acceleration in the x-axis direction fluctuates around 0.
[0151] Frames 100 to 260 are the process of flipping the screen from the vertical position at the back to the horizontal position at the front. During this flipping process, the gravitational acceleration along the x-axis, y-axis, and z-axis and the rotational angular velocity along the x-axis, y-axis, and z-axis all change significantly.
[0152] The 260th to 360th frames are the horizontal stable state of the screen. Among them, the value of the gravity acceleration in the x-axis direction is close to 9.8, and the values of the rotational angular acceleration in the x-axis, y-axis and z-axis are all approximately equal to 0.
[0153] See also Fig.12 , Fig.12 is based on Figure 4The illustrated embodiment provides another schematic diagram of raw sample data collected by a terminal. Fig.12 What is shown is the state where the rear vertical position of the screen is flipped to the front vertical position of the screen. Fig.12 For the reference numerals and their meanings, see Figure 7 The content shown.
[0154] exist Fig.12 In the scene shown, the 0th frame to the 150th frame is a stable state of the terminal screen back vertically, wherein the value of the gravity acceleration in the y-axis direction is greater than 5, the value of the gravity acceleration in the z-axis direction fluctuates around 5, and the gravity acceleration in the x-axis direction fluctuates around 0.
[0155] Frames 150 to 230 are the process of flipping the screen from the back vertical to the front vertical. During this flipping process, the gravitational acceleration of the three axes, x-axis, y-axis and z-axis, and the rotational angular velocity of the three axes, x-axis, y-axis and z-axis, all change significantly.
[0156] The 230th to 360th frames are the stable state of the screen in the vertical direction. Among them, the value of the gravity acceleration in the y-axis direction is close to 9.8, and the values of the rotational angular acceleration in the three axes of the x-axis, y-axis and z-axis are all approximately equal to 0.
[0157] See also Fig.13 , Fig.13 is based on Figure 4 The illustrated embodiment provides another schematic diagram of raw sample data collected by a terminal. Fig.13 The screen is shown as being flipped vertically to face upward. Fig.13 For the reference numerals and their meanings, see Figure 7 The content shown.
[0158] exist Fig.13 In the scene shown, the 0th frame to the 150th frame is a stable state of the terminal screen back vertically, wherein the value of the gravity acceleration in the y-axis direction is greater than 5, the value of the gravity acceleration in the z-axis direction fluctuates around 5, and the gravity acceleration in the x-axis direction fluctuates around 0.
[0159] Frames 150 to 230 are the process of the screen flipping from the back to the top. During this flipping process, the gravitational acceleration along the x-axis, y-axis, and z-axis and the rotational angular velocity along the x-axis, y-axis, and z-axis all change significantly.
[0160] The 230th to 360th frames are in a stable state with the screen pointing upwards. The value of the gravity acceleration in the z-axis direction is close to 9.8, and the values of the rotational angular accelerations in the x-axis, y-axis, and z-axis directions are all approximately equal to 0.
[0161] Optionally, after the pre-labeling function is integrated into the collection tool, when collecting the original sample data, the user only needs to click to start the detection, and then start to rotate the terminal as required. After a few seconds, the terminal will stop detecting and display the sensor data fitting curves of the start sampling frame and the end sampling frame on the screen. In this embodiment, the start sampling frame automatically detected by the pre-labeling function is the 33rd frame, the stable frame is the 98th frame, and the end sampling frame is the 116th frame. It should be noted that the value specified by j here is 18. It can be seen that the terminal can check the starting action and ending action, the starting sampling frame and the ending sampling frame of the terminal by itself. If there is an error in the above-mentioned automation process, the user can modify it directly in the interface. After the modification is completed, click the write data button to complete the acquisition process of the target sample once.
[0162] To sum up, this embodiment can determine the starting sampling frame from the sensor fitting data according to the first change of the sensor data, and determine the sampling frame of the second duration after the starting sampling frame as the ending sampling frame based on the starting sampling frame, thereby achieving the effect of automatically intercepting the target sample from the original sampling frame data and improving the efficiency of automatically acquiring the target sample.
[0163] Optionally, the sample acquisition method provided in this embodiment can also determine the end sampling frame from the sensor fitting data according to the second change of the sensor data, and use the end sampling frame as a reference to determine the sampling frame of the second duration before the end sampling frame as the start sampling frame, thereby achieving the effect of automatically intercepting the target sample from the original sampling frame data and improving the efficiency of automatically acquiring the target sample.
[0164] Optionally, the sample acquisition method provided in this embodiment can also determine the start sampling frame and the end sampling frame according to the first change situation and the second change situation of the sensor data, so that the terminal can automatically extract the target sample from the original sample data.
[0165] Optionally, the sample acquisition method provided in this embodiment can be applied not only to terminals dedicated to collecting sample data, but also to terminals used by users in daily life. In the scenario where the user presents a boarding code, payment code or health code, the original sample data is automatically collected, and the target sample is determined from the original sample data, thereby increasing the data channels for acquiring samples and improving the efficiency of acquiring target samples, so that the machine learning model that identifies the flipping action generated by the terminal can be recognized more quickly.
[0166] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0167] Please refer to Fig.14 , Fig.14 This is a structural block diagram of a sample acquisition device provided by an exemplary embodiment of the present application. The sample acquisition device can be implemented as all or part of a terminal through software, hardware, or a combination of both. The device includes:
[0168] The data collection module 810 is used to collect raw sample data of a first duration, where the raw sample data is used to indicate a posture change of the terminal within the first duration;
[0169] A frame determination module 820, configured to determine a start sampling frame and an end sampling frame in the original sample data, wherein the sampling frame between the start sampling frame and the end sampling frame is used to indicate a process of a flipping action performed by the terminal;
[0170] The sample acquisition module 830 is used to intercept the sampling frames between the start sampling frame and the end sampling frame as target samples, and the target samples are used to train a machine learning model, and the machine learning model is used to identify the flipping action generated by the terminal.
[0171] In an optional embodiment, the frame determination module 820 is used to determine a start sampling frame based on a first change situation of the sensor data, and determine a sampling frame of a second duration after the start sampling frame as an end sampling frame, wherein the first change situation is used to indicate that the change rate of the sensor data is greater than a first threshold; or, determine an end sampling frame based on a second change situation of the sensor data, and determine a sampling frame of a second duration before the end sampling frame as a start sampling frame, wherein the second change situation is used to indicate that the change rate of the sensor data is less than a second threshold; or, determine a start sampling frame based on the first change situation of the sensor data, and determine an end sampling frame based on the second change situation of the sensor data, wherein the first change situation is used to indicate that the change rate of the sensor data is greater than the first threshold, and the second change situation is used to indicate that the change rate of the sensor data is less than the second threshold.
[0172] In an optional embodiment, the frame determination module 820 is used to determine the first sampling frame in the original sample data as the start sampling frame in response to the absolute value of the slope of the sensor data fitting curve at the first sampling frame being less than the first threshold, and the absolute value of the slope of the sensor data fitting curve at the second sampling frame being greater than or equal to the first threshold, the sensor data fitting curve being a curve fitted according to the sensor data in each sampling frame, the first sampling frame being an adjacent sampling frame whose collection time is earlier than the second sampling frame; determine the sampling frame whose collection time is later than the start sampling frame by a second time length as the end sampling frame; or, in response to the absolute value of the slope of the sensor data fitting curve at the third sampling frame being less than the second threshold, and the absolute value of the slope of the sensor data fitting curve at the fourth sampling frame being greater than or equal to the second threshold, determine the fourth sampling frame in the original sample data as the end sampling frame, the sensor data fitting curve being a curve fitted according to the sensor data in each sampling frame, The third sampling frame is an adjacent sampling frame whose collection time is earlier than that of the fourth sampling frame; a sampling frame whose collection time is earlier than that of the end sampling frame by a second duration is determined as the start sampling frame; or, in response to the absolute value of the slope of the sensor data fitting curve at the first sampling frame being less than the first threshold, and the absolute value of the slope of the sensor data fitting curve at the second sampling frame being greater than or equal to the first threshold, the first sampling frame in the original sample data is determined as the start sampling frame, the sensor data fitting curve is a curve fitted according to the sensor data in each sampling frame, and the first sampling frame is an adjacent sampling frame whose collection time is earlier than that of the second sampling frame; in response to the absolute value of the slope of the sensor data fitting curve at the third sampling frame being less than the second threshold, and the absolute value of the slope of the sensor data fitting curve at the fourth sampling frame being greater than or equal to the second threshold, the fourth sampling frame in the original sample data is determined as the end sampling frame, the third sampling frame being an adjacent sampling frame whose collection time is earlier than that of the fourth sampling frame.
[0173] In an optional embodiment, the sensor data involved in the device includes any one of gravitational acceleration in the x-axis direction, gravitational acceleration in the y-axis direction, gravitational acceleration in the z-axis direction, angular acceleration of rotation in the x-axis direction, angular acceleration of rotation in the y-axis direction, and angular acceleration of rotation in the z-axis direction.
[0174] In an optional embodiment, the frame determination module 820 is used to traverse the sampling frames in the original sample data in sequence according to the acquisition order, and detect whether the posture of the terminal corresponding to the sampling frame meets the preset posture; in response to the posture of the terminal corresponding to the sampling frame meeting the preset posture, set a counter, and the initial value of the counter is zero; in response to the posture of the terminal corresponding to the sampling frame detected after the sampling frame meeting the preset posture, the value of the counter is increased by one; in response to the value of the counter being equal to n, the sampling frame is determined as the end sampling frame; and the sampling frame whose acquisition time is earlier than the second time length of the end sampling frame is determined as the start sampling frame.
[0175] In an optional embodiment, the frame determination module 820 is used to obtain the posture type of the preset posture; determine the value range of the corresponding sensor data according to the posture type, and the sensor data includes at least one of gravitational acceleration or rotational angular acceleration; detect whether the sensor data corresponding to the sampling frame conforms to the value range of the sensor data.
[0176] In an optional embodiment, the frame determination module 820 is used to detect, in response to the sensor data including the gravity acceleration, whether the measured value of the gravity acceleration along a first direction is equal to a constant value of the gravity acceleration, wherein the first direction includes a normal direction outward along the screen of the terminal; or, in response to the sensor data including the gravity acceleration and the rotational angular acceleration, detect whether the measured value of the gravity acceleration along the first direction is equal to the constant value of the gravity acceleration, and detect whether the absolute value of the rotational angular acceleration is less than a preset threshold.
[0177] In an optional embodiment, the device also includes a reset module for resetting the counter in response to the posture of the terminal corresponding to the sampling frame detected after the sampling frame not conforming to the preset posture; the frame determination module 820 is used to traverse the sampling frames in the original sample data in the order of acquisition for the sampling frames that have not been detected in the m sampling frames, and detect whether the posture of the terminal corresponding to the sampling frame conforms to the preset posture.
[0178] In an optional embodiment, the device also includes a first signal receiving module and a first posture determination module, the first signal receiving module is used to receive a first trigger signal, the first trigger signal is a signal generated when the first control of the target application in the terminal is triggered; the first posture determination module is used to determine the preset posture of the terminal corresponding to the start sampling frame according to the first trigger signal, or determine the preset posture corresponding to the end sampling frame.
[0179] In an optional embodiment, the data acquisition module 810 is used to receive a second trigger signal, which is a signal generated when a second control in the target application in the terminal is triggered; taking the moment when the second trigger signal is received as the starting moment, the sensor collects the original sample data for the first duration; wherein the second control is at least one of a data collection button in the sample collection application, a payment code display button in the payment application, a boarding code display button in the public transportation application, or a health code display button in the health application.
[0180] In summary, the device provided by the embodiment of the present application can collect raw sample data of a first duration through a sensor in a terminal, and after obtaining the raw sample data, determine the start sampling frame and the end sampling frame from the raw sample, wherein the sampling frame between the start sampling frame and the end sampling frame is used to indicate a flipping action process performed by the terminal, and the sampling frame between the start sampling frame and the end sampling frame is intercepted as a target sample to train a machine learning model that can identify the flipping action generated by the terminal. Since the present application can automatically determine the start sampling frame and the end sampling frame in the raw sample data when collecting the target sample, the step of manually confirming the above-mentioned frames is reduced, thereby improving the efficiency of obtaining the target sample.
[0181] An embodiment of the present application further provides a computer-readable medium storing at least one instruction, wherein the at least one instruction is loaded and executed by the processor to implement the sample acquisition method described in the above embodiments.
[0182] It should be noted that: when the sample acquisition device provided in the above embodiment executes the sample acquisition method, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the sample acquisition device provided in the above embodiment and the sample acquisition method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0183] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0184] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0185] The above description is only an illustrative embodiment that can be implemented in the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.
Claims
1. A method for obtaining a sample, characterized in that: The method comprises: receiving a first trigger signal, where the first trigger signal is a signal generated when a first control of a target application in the terminal is triggered, and different first controls correspond to different preset postures; Determine the preset posture corresponding to the end sampling frame according to the first trigger signal, and the sampling frame between the start sampling frame and the end sampling frame is used to indicate a process of a flipping action performed by the terminal; Collecting original sample data having a first duration, wherein the original sample data is used to indicate a change in the posture of the terminal within the first duration; Traversing the sampling frames in the original sample data in sequence according to the acquisition order, and detecting whether the posture of the terminal corresponding to the sampling frame conforms to the preset posture; In response to the posture of the terminal corresponding to the first sampling frame conforming to the preset posture, setting a counter, wherein an initial value of the counter is zero; In response to the posture of the terminal corresponding to the sampling frame detected after the first sampling frame conforming to the preset posture, the value of the counter is increased by one; In response to the posture of the terminal corresponding to the sampling frame detected after the first sampling frame not meeting the preset posture, resetting the counter; For the sampling frames that have not been detected in the m sampling frames, re-execute the step of sequentially traversing the sampling frames in the original sample data in the acquisition order, and detecting whether the posture of the terminal corresponding to the sampling frame meets the preset posture; In response to the value of the counter being equal to n, determining the corresponding sampling frame as the end sampling frame; Determine a sampling frame whose acquisition time is earlier than the second duration of the end sampling frame as the start sampling frame; The sampling frames between the start sampling frame and the end sampling frame are intercepted as target samples, and the target samples are used to train a machine learning model, and the machine learning model is used to identify the flipping action generated by the terminal.
2. The method according to claim 1, characterized in that The detecting whether the posture of the terminal corresponding to the sampling frame conforms to the preset posture includes: Obtaining the posture type of the preset posture; Determining a value interval of corresponding sensor data according to the posture type, wherein the sensor data includes at least one of gravitational acceleration or rotational angular acceleration; Detect whether the sensor data corresponding to the sampling frame conforms to the value interval of the sensor data.
3. The method according to claim 2, characterized in that The detecting whether the sensor data corresponding to the sampling frame conforms to the value interval of the sensor data includes: In response to the sensor data including the gravity acceleration, detecting whether a measured value of the gravity acceleration along a first direction is equal to a constant value of the gravity acceleration, the first direction including a normal direction outward from a screen of the terminal; or, In response to the sensor data including the gravitational acceleration and the rotational angular acceleration, detecting whether a measured value of the gravitational acceleration along a first direction is equal to a constant value of the gravitational acceleration, and detecting whether an absolute value of the rotational angular acceleration is less than a preset threshold.
4. The method according to any one of claims 1 to 3, characterized in that: The original sample data whose collection duration is the first duration includes: receiving a second trigger signal, where the second trigger signal is a signal generated when a second control in a target application in the terminal is triggered; Taking the time of receiving the second trigger signal as the starting time, collecting the original sample data having the first time length through the sensor; Among them, the second control is at least one of a data collection button in a sample collection application, a payment code display button in a payment application, a boarding code display button in a public transportation application, or a health code display button in a health application.
5. A sample acquisition device, characterized in that: The device comprises: A first signal receiving module, configured to receive a first trigger signal, wherein the first trigger signal is a signal generated when a first control of a target application in the terminal is triggered, and different first controls correspond to different preset gestures; A first posture determination module, used to determine the preset posture corresponding to the end sampling frame according to the first trigger signal, and the sampling frame between the start sampling frame and the end sampling frame is used to indicate a process of a flipping action performed by the terminal; A data collection module, used for collecting raw sample data of a first duration, wherein the raw sample data is used for indicating a change in the posture of the terminal within the first duration; A frame determination module, configured to sequentially traverse the sampling frames in the original sample data in the order of acquisition, and detect whether the posture of the terminal corresponding to the sampling frame conforms to the preset posture; in response to the posture of the terminal corresponding to the first sampling frame conforming to the preset posture, set a counter, the initial value of the counter being zero; in response to the posture of the terminal corresponding to the sampling frame detected after the first sampling frame conforming to the preset posture, the value of the counter is increased by one; A reset module, configured to reset the counter in response to the posture of the terminal corresponding to the sampling frame detected after the first sampling frame not conforming to the preset posture; The frame determination module is used to traverse the sampling frames in the original sample data in the order of acquisition for the sampling frames that have not been detected in the m sampling frames, and detect whether the posture of the terminal corresponding to the sampling frame meets the preset posture; in response to the value of the counter being equal to n, determine the corresponding sampling frame as the end sampling frame; determine the sampling frame whose acquisition time is earlier than the second time length of the end sampling frame as the start sampling frame; A sample acquisition module is used to intercept the sampling frames between the start sampling frame and the end sampling frame as target samples, and the target samples are used to train a machine learning model, and the machine learning model is used to identify the flipping action generated by the terminal.
6. A terminal, characterized in that: The terminal includes a processor, a memory connected to the processor, and program instructions stored in the memory. When the processor executes the program instructions, the sample acquisition method according to any one of claims 1 to 4 is implemented.
7. A computer-readable storage medium, wherein program instructions are stored in the storage medium, characterized in that: When the program instructions are executed by a processor, the sample acquisition method according to any one of claims 1 to 4 is implemented.
Citation Information
Patent Citations
Method and device for invoking target function, mobile terminal and storage medium
CN109324737A
Sample data acquisition method and device, electronic equipment and storage medium
CN112817452A
Terminal screen shaking identification method and apparatus
WO2017036205A1
