Feature extraction method, device, storage medium and electronic device
By combining geographical location and time information in feature extraction, the cross feature set is generated, and the problem of insufficient feature dimensions in the prior art is solved, and the accuracy of scenario category recognition is improved.
Patent Information
- Application Number
- CN201910282476.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-04-09
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2039-04-09
AI Technical Summary
In the prior art, the feature extraction scheme extracts few feature dimensions from panoramic data and the information is not comprehensive enough, resulting in low accuracy in scenario category recognition.
By collecting panoramic data based on geographical location and time information, extracting feature sets respectively, and cross-fusion them to generate cross-feature sets based on geographical location and time information, increasing feature dimensions and scales, and fusing geographical location attributes and time attributes.
It improves the accuracy of scenario category recognition, can better characterize users' behavioral habits and preferences, and improves the accuracy of scenario category recognition.
Smart Images

Figure CN111797261B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of feature extraction technology, and in particular to a feature extraction method, device, storage medium and electronic device. Background Art
[0002] If the terminal scene is identified by collecting terminal panoramic data, it is necessary to process the collected panoramic data to extract panoramic features from it, and use the panoramic features to characterize or identify the current scene category. However, most feature extraction solutions in the prior art extract panoramic features separately from existing data, which face problems such as few feature dimensions and insufficient information, resulting in low accuracy in identifying scene categories. Summary of the invention
[0003] The embodiments of the present application provide a feature extraction method, device, storage medium and electronic device, which can increase the feature dimension of scene recognition and improve the recognition accuracy of scene categories.
[0004] In a first aspect, an embodiment of the present application provides a feature extraction method, comprising:
[0005] Collecting first panoramic data based on the geographical location;
[0006] Generate a first feature set according to a preset first feature extraction algorithm and the first panoramic data;
[0007] collecting second panoramic data based on time information;
[0008] Generate a second feature set according to a preset second feature extraction algorithm and the second panoramic data;
[0009] The first feature set and the second feature set are cross-fused to generate a cross feature set based on geographic location and time information.
[0010] In a second aspect, an embodiment of the present application provides a feature extraction device, comprising:
[0011] A first data collection module, used to collect first panoramic data based on the geographical location;
[0012] A first feature extraction module, used for generating a first feature set according to a preset first feature extraction algorithm and the first panoramic data;
[0013] A second data acquisition module, used for acquiring second panoramic data based on time information;
[0014] A second feature extraction module, used for generating a second feature set according to a preset second feature extraction algorithm and the second panoramic data;
[0015] The feature fusion module is used to cross-fuse the first feature set and the second feature set to generate a cross feature set based on geographic location and time information.
[0016] In a third aspect, the storage medium provided in the embodiments of the present application stores a computer program thereon, and when the computer program is run on a computer, the computer executes a feature extraction method provided in any embodiment of the present application.
[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein the memory has a computer program, and the processor executes a feature extraction method provided in any embodiment of the present application by calling the computer program.
[0018] The technical solution provided in the embodiment of the present application collects panoramic data based on geographic location and time information respectively, and for the first panoramic data collected based on the geographic location, features are extracted therefrom to form a first feature set; at the same time, for the second panoramic data collected based on the time information, features are extracted therefrom to form a second feature set, and the first feature set and the second feature set are cross-fused to generate a cross feature set based on geographic location and time information, which not only increases the dimension and scale of the features obtained from the panoramic data, but also can fuse the geographic location attributes and time attributes to make the features more comprehensive, and in the identification of scenario categories, can better characterize the user's behavioral habits and preferences, thereby improving the accuracy of scenario category identification. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0020] Figure 1 A schematic diagram of a panoramic perception architecture of a feature extraction method provided in an embodiment of the present application.
[0021] Figure 2 A first flow chart of the feature extraction method provided in the embodiment of the present application.
[0022] Figure 3 A second flow chart of the feature extraction method provided in the embodiment of the present application.
[0023] Figure 4 A third flow chart of the feature extraction method provided in the embodiment of the present application.
[0024] Figure 5A schematic diagram of the structure of a feature extraction device provided in an embodiment of the present application.
[0025] Figure 6 A first structural schematic diagram of an electronic device provided in an embodiment of the present application.
[0026] Figure 7 A second structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0028] refer to Figure 1 , Figure 1 A schematic diagram of a panoramic perception architecture of a feature extraction method provided in an embodiment of the present application. The feature extraction method is applied to an electronic device. A panoramic perception architecture is provided in the electronic device. The panoramic perception architecture is an integration of hardware and software in the electronic device for implementing the feature extraction method.
[0029] Among them, the panoramic perception architecture includes information perception layer, data processing layer, feature extraction layer, scenario modeling layer and intelligent service layer.
[0030] The information perception layer is used to obtain information about the electronic device itself or information in the external environment. The information perception layer may include multiple sensors. For example, the information perception layer includes multiple sensors such as distance sensors, magnetic field sensors, light sensors, acceleration sensors, fingerprint sensors, Hall sensors, position sensors, gyroscopes, inertial sensors, attitude sensors, barometers, heart rate sensors, etc.
[0031] Among them, the distance sensor can be used to detect the distance between the electronic device and the external object. The magnetic field sensor can be used to detect the magnetic field information of the environment in which the electronic device is located. The light sensor can be used to detect the light information of the environment in which the electronic device is located. The acceleration sensor can be used to detect the acceleration data of the electronic device. The fingerprint sensor can be used to collect the fingerprint information of the user. The Hall sensor is a magnetic field sensor made according to the Hall effect, which can be used to realize the automatic control of the electronic device. The position sensor can be used to detect the current geographical location of the electronic device. The gyroscope can be used to detect the angular velocity of the electronic device in all directions. The inertial sensor can be used to detect the motion data of the electronic device. The attitude sensor can be used to sense the attitude information of the electronic device. The barometer can be used to detect the air pressure of the environment in which the electronic device is located. The heart rate sensor can be used to detect the heart rate information of the user.
[0032] The data processing layer is used to process the data obtained by the information perception layer. For example, the data processing layer can perform data cleaning, data integration, data transformation, data reduction, etc. on the data obtained by the information perception layer.
[0033] Among them, data cleaning refers to cleaning the large amount of data obtained by the information perception layer to eliminate invalid data and duplicate data. Data integration refers to integrating multiple single-dimensional data obtained by the information perception layer into a higher or more abstract dimension to comprehensively process multiple single-dimensional data. Data transformation refers to converting the data type or format of the data obtained by the information perception layer so that the transformed data meets the processing requirements. Data reduction refers to maximizing the reduction of data volume while keeping the original appearance of the data as much as possible.
[0034] The feature extraction layer is used to extract features from the data processed by the data processing layer to extract features included in the data. The extracted features can reflect the state of the electronic device itself, the state of the user, or the environmental state of the environment in which the electronic device is located.
[0035] Among them, the feature extraction layer can extract features or process the extracted features through filtering, packaging, integration and other methods.
[0036] The filtering method refers to filtering the extracted features to delete redundant feature data. The packaging method is used to screen the extracted features. The integration method refers to integrating multiple feature extraction methods together to build a more efficient and accurate feature extraction method for extracting features.
[0037] The scenario modeling layer is used to construct a model based on the features extracted by the feature extraction layer, and the obtained model can be used to represent the state of the electronic device, the state of the user, or the state of the environment, etc. For example, the scenario modeling layer can construct a key value model, a pattern identification model, a graph model, an entity relationship model, an object-oriented model, etc. based on the features extracted by the feature extraction layer.
[0038] The intelligent service layer is used to provide intelligent services to users based on the model constructed by the scenario modeling layer. For example, the intelligent service layer can provide basic application services to users, perform system intelligent optimization for electronic devices, and provide personalized intelligent services to users.
[0039] In addition, the panoramic perception architecture may also include multiple algorithms, each of which can be used to analyze and process data, and the multiple algorithms may constitute an algorithm library. For example, the algorithm library may include Markov algorithm, invisible Dirichlet distribution algorithm, Bayesian classification algorithm, support vector machine, K-means clustering algorithm, K-nearest neighbor algorithm, conditional random field, residual network, long short-term memory network, convolutional neural network, recurrent neural network and other algorithms.
[0040] Based on the above panoramic perception framework, the electronic device collects the first panoramic data based on the geographic location and the second panoramic data based on the time information through the information perception layer and / or other methods, and then the data processing layer processes the first panoramic data and the second panoramic data, such as row data cleaning, data integration, etc. Next, the feature extraction layer performs feature extraction according to the feature extraction scheme provided in the embodiment of the present application, for example, the panoramic data is collected based on the geographic location and time information respectively, and for the first panoramic data collected based on the geographic location, features are extracted from it to form a first feature set; at the same time, for the second panoramic data collected based on the time information, features are extracted from it to form a second feature set, and the first feature set and the second feature set are cross-fused to generate a cross-feature set based on the geographic location and time information, which not only increases the dimension and scale of the features obtained from the panoramic data, but also can fuse the geographic location attributes and time attributes, making the features more comprehensive, and in the recognition of scenario categories, can better characterize the user's behavior habits and preferences, thereby improving the accuracy of scenario category recognition.
[0041] The embodiment of the present application provides a feature extraction method, and the execution subject of the feature extraction method can be a feature extraction device provided in the embodiment of the present application, or an electronic device integrating the feature extraction device, wherein the feature extraction device can be implemented in hardware or software. The electronic device can be a smart phone, a tablet computer, a PDA, a laptop computer, or a desktop computer.
[0042] Taking the feature extraction device integrated in the smart terminal as an example, the electronic device can collect panoramic data based on geographic location and time information respectively. For the first panoramic data collected based on the geographic location, features are extracted from it to form a first feature set; at the same time, for the second panoramic data collected based on the time information, features are extracted from it to form a second feature set. The first feature set and the second feature set are cross-fused to generate a cross feature set based on geographic location and time information. This not only increases the dimension and scale of the features obtained from the panoramic data, but also can fuse geographic location attributes and time attributes to make the features more comprehensive. In the identification of scenario categories, it can better characterize the user's behavioral habits and preferences, thereby improving the accuracy of scenario category identification.
[0043] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the first process of the feature extraction method provided in the embodiment of the present application. The specific process of the feature extraction method provided in the embodiment of the present application can be as follows:
[0044] Step 101: Collect first panoramic data based on the geographical location.
[0045] The first panoramic data, the second panoramic data, etc. in this embodiment include sensor data. The sensor data includes signals collected by various sensors on the electronic device. For example, the electronic device is provided with the following sensors: distance sensor, magnetic field sensor, light sensor, acceleration sensor, fingerprint sensor, Hall sensor, position sensor, gyroscope, inertial sensor, attitude sensor, barometer, heart rate sensor, etc.
[0046] In some embodiments, the state data of some sensors may be obtained in a targeted manner. For example, data collected by a position sensor and a light sensor may be obtained, wherein the current position information of the electronic device may be determined based on the data collected by the position sensor, and the light sensor may collect the light intensity of the environment in which the electronic device is currently located.
[0047] Furthermore, in some embodiments, the panoramic data may also include the following categories: environmental data, user behavior data, and terminal operation data. Each category of data includes multiple data items, and the panoramic data is a collection of all data items. In addition, the "first" and "second" in the first panoramic data and the second panoramic data are only used to distinguish the panoramic data obtained based on different information, and they contain data items of the same category.
[0048] Among them, environmental data includes time, location, air quality, weather, temperature, humidity, sound, light and other data. Some of the above data can be collected through corresponding sensors on electronic devices, such as time and location, and another part of the data can be obtained through the network after determining the time and location of the electronic device.
[0049] User behavior data includes the history of users launching applications, the history of users listening to music, the history of users watching videos, the history of users calling, the history of users playing games, and other data. The usage records of related applications are collected through electronic devices.
[0050] The terminal operation data includes the operation mode of the electronic device in each time interval. For example, the operation mode includes game mode, entertainment mode, audio and video mode, etc. The operation mode of the electronic device can be determined according to the type of the currently running application. The type of the currently running application can be directly obtained from the classification information of the application installation package; or, the terminal operation data can also include the remaining power of the electronic device, display mode, network status, screen off / locked status, etc.
[0051] In the following, panoramic data is used as sensor data as an example. When a user uses an electronic device, there will be multiple different locations. When the user is in different geographical locations, the application scenarios of the electronic device are also different, so the sensor data used is also different. In addition, the application scenarios of the electronic device by the user in different time periods are also different, resulting in certain time characteristics of the sensor data used.
[0052] like Figure 3 As shown, Figure 3 A second flow chart of the feature extraction method provided in the embodiment of the present application. In some embodiments, step S101, the step of collecting first panoramic data based on the geographical location, includes:
[0053] Step S1011: collecting first panoramic data at a preset frequency, and recording the geographical location corresponding to each piece of first panoramic data;
[0054] Step S1012: Taking the geographical location as a reference, statistics are collected on the first panoramic data to obtain the first panoramic data corresponding to each geographical location.
[0055] During operation, the electronic device continuously collects sensor data as first panoramic data at a preset frequency. It is predetermined which sensor data is to be collected, and when recording the collected sensor data, the sensor data is taken as one piece of data, that is, one piece of first panoramic data includes data of multiple sensors. Each piece of sensor data corresponds to a piece of geographic location information, and when recording a piece of sensor data, the geographic location information when the piece of sensor data is collected is associated with the piece of sensor data and recorded.
[0056] For example, MySQL is used in advance to build a panoramic database based on geographic location. After collecting sensor data, the electronic device counts the collected sensor data regularly or at preset time intervals, for example, once every 24 hours, based on the geographic location, and obtains the sensor data corresponding to each geographic location. The sensor data is stored in the above MySQL database in the following form: i ,d(a i )>, i∈(1,n).
[0057] Among them, a i Indicates geographic location, d(a i ) indicates that at geographic location a i The size of n is not specific, but is determined based on the actual number of geographical locations after counting the collected sensor data based on the geographical location.
[0058] In addition, the location information obtained by locating electronic devices through location sensors is generally longitude and latitude data. After a period of collection, a lot of different longitude and latitude data will be generated. Each longitude and latitude data represents a positioning point, and in fact, the distance between some positioning points is very close. Therefore, a clustering algorithm can be used to statistically analyze the collected sensor data based on the geographic location.
[0059] Step 102: Generate a first feature set according to a preset first feature extraction algorithm and the first panoramic data.
[0060] In some embodiments, taking the geographical location as a reference, the collected first panoramic data is counted, and the step of obtaining the first panoramic data corresponding to each geographical location includes:
[0061] Counting the geographical locations of the first panoramic data and generating a geographical location list;
[0062] Determining a density parameter for cluster analysis, clustering the geographical locations in the geographical location list according to the density parameter and a preset clustering algorithm, and acquiring a plurality of geographical location sets;
[0063] Based on the geographical location set, statistics are performed on the collected first panoramic data to obtain the first panoramic data corresponding to each geographical location set.
[0064] The preset clustering algorithm may be a DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm. The density parameters are the aggregation radius and MinPts, where MinPts is the minimum number of clusters. These two parameters may be preset as needed, for example, the aggregation radius may be set to 1 km and MinPts may be set to 10.
[0065] The geographical location corresponding to each first panoramic data of the record is obtained, a geographical location list is generated, and the geographical locations in the list are clustered according to the DBSCAN algorithm, and finally a set of multiple geographical locations is generated, each set contains multiple longitude and latitude data, among which the more central longitude and latitude data in a geographical location set can be selected to represent the geographical location set.
[0066] For example, in some embodiments, after obtaining multiple geographic location sets through the DBSCAN algorithm, the geographic location data in each set is used as a new input, and the position of the centroid is obtained by iterative aggregation using the Kmeans algorithm, and the K value is set to 1. After the centroid is obtained, the centroid is used as the center point of the geographic location set, the longitude and latitude data of the center point are obtained, and its address on the map is found to represent the geographic location set. For example, No. XX, XX Street, XX District, XX City.
[0067] In this way, the centroid of each geographic location set can be obtained, and the specific address information can be obtained. Assuming there are 10 geographic location sets, we can finally obtain a1, a2, ... a 10 A total of ten specific address information.
[0068] Next, taking the geographical location set as a unit, the first panoramic data corresponding to each geographical location in the geographical location set is obtained, and after associating the first panoramic data with the address of the centroid of the geographical location set, the first panoramic data is obtained. i ,d(a i For example, a1 is the centroid of a clustered geographical location set, representing all geographical locations in the geographical location set, and d(a1) represents the first panoramic data corresponding to all geographical locations in the geographical location set.
[0069] Among them, d(a) can be expressed in the form of a third-order tensor i ), the third-order tensor has three dimensions: the geographic location corresponds to one dimension. In addition, since a geographic location will collect multiple first panoramic data, the number of first panoramic data corresponds to one dimension. Moreover, since there are many types of panoramic data, the types of panoramic data constitute the third dimension.
[0070] After the panoramic data collection is completed, the first panoramic data based on geographic location statistics stored in the MySQL database is obtained. i ,d(a i )>, perform feature extraction according to a preset first feature extraction algorithm to generate a first feature set.
[0071] Among them, if the DBSCAN algorithm is used to cluster the geographic locations, then step 102, the step of generating a first feature set according to a preset first feature extraction algorithm and the first panoramic data, may include the following detailed steps: according to the preset first feature extraction algorithm, obtaining the first feature corresponding to the geographic location set from the corresponding first panoramic data, and the first feature set is composed of the first features corresponding to the multiple geographic location sets.
[0072] In the embodiments of the present application, the following two methods for extracting the first feature based on geographic location can be provided. Method 1: According to a feature extraction algorithm based on statistics, statistical features are extracted from the first panoramic data corresponding to each geographic location to form the first feature set. The feature extraction algorithm based on statistics generally extracts the maximum value, minimum value, mean value and other data in each type of first panoramic data as features. For example, if the first panoramic data includes sensor data, the data of each sensor is taken as a type of first panoramic data to extract features. For example, for<a1,d(a1)> For the acceleration sensor data in d(a1), all the acceleration sensor data in d(a1) are obtained. Assuming that a total of 100 acceleration sensor data are recorded, the maximum value, minimum value, and mean value of these 100 acceleration sensor data are calculated as features.
[0073] In this way, all features corresponding to a1 can be extracted to form the first feature, which can be expressed as f(a1). i The corresponding f(a i ), where i∈(1,n), assuming there are 10 sensors in total, each of which corresponds to three eigenvalues: maximum, minimum, and mean. Then f(a i ) can be represented as a long vector of length 30.
[0074] The first feature set based on geographic location can be expressed as:
[0075] X=[f(a1),f(a2),……,f(a i ),……f(a n )].
[0076] Method 2: According to a preset first feature extraction algorithm and the first panoramic data, the step of generating a first feature set includes: converting the first panoramic data into frequency domain panoramic data according to a discrete Fourier transform; extracting frequency domain features from the frequency domain panoramic data corresponding to each geographical location to form the first feature set X = [f(a1), f(a2), ..., f(a n )].
[0077] In the second method, discrete Fourier transform is used to convert the discrete first panoramic data into frequency domain panoramic data, and then frequency domain features such as amplitude, period, phase and other frequency domain features are extracted. Here, feature extraction is also performed in units of sensor type, that is, for each sensor, the above-mentioned frequency domain features such as amplitude, period, phase and the like can be obtained in the above-mentioned manner. After feature extraction is performed on all sensor data in a geographic location set, a large number of first features are generated to form a first feature set X.
[0078] Step 103: Collect second panoramic data based on the time information.
[0079] Step 104: Generate a second feature set according to a preset second feature extraction algorithm and the second panoramic data.
[0080] Next, the second panoramic data is collected based on the time information. Since time is sequential, when the panoramic data collected based on the time information is counted, time periods can be divided and information statistics can be performed according to the time periods.
[0081] Reference Figure 4 As shown, Figure 4 A third flow chart of the feature extraction method provided in the embodiment of the present application. In some embodiments, step 103, the step of collecting the second panoramic data based on the time information, includes:
[0082] Step 1031: collect the second panoramic data within the time interval according to a preset frequency, and record the time information of each piece of the second panoramic data;
[0083] Step 1032: Divide the time interval into multiple time periods;
[0084] Step 1033: Based on the time period, statistics are collected on the second panoramic data within the time interval to obtain a second panoramic data sequence corresponding to each time period.
[0085] In which, 24 hours is taken as a time interval, and the sensor in the electronic device will collect the second panoramic data in the time interval according to the preset frequency in this time interval, and the sensor data to be collected is predetermined, and when the collected sensor data is recorded, these sensor data are taken as one piece of data, that is, one piece of second panoramic data includes data of multiple sensors. The collection time of each piece of sensor data is recorded as the corresponding time information.
[0086] Next, the time interval is divided into multiple time periods, and the panoramic data within the time interval is counted according to the time periods.
[0087] In some embodiments, the time periods may be divided manually, for example, each hour is regarded as a time period, and a day may be divided into 24 time periods.
[0088] Alternatively, in some other embodiments, the time period may be divided based on information entropy. Specifically, step 1032, dividing the time interval into a plurality of time periods, includes:
[0089] Calculating the total information entropy of the time interval according to the second panoramic data in the time interval;
[0090] According to a time division point in a preset value range, the time interval is divided into two time periods, and the weighted average information entropy of the two divided time periods is calculated;
[0091] Determine the time segmentation point at which the difference between the weighted average information entropy and the total information entropy is the largest;
[0092] Dividing the time interval into a first time period and a second time period according to the time division point;
[0093] Taking the first time period and the second time period as new time intervals, repeat the step of calculating the total information entropy of the time interval based on the second panoramic data within the time interval to the step of dividing the time interval into the first time period and the second time period according to the time division point, until the number of segments of the time interval is greater than a preset threshold.
[0094] Information entropy can be calculated using any type of second panoramic data. For example, the second panoramic data includes terminal operation data, which includes APP startup data. Taking WeChat APP as an example, assuming that the time unit is 10 minutes, there are a total of 144 time units in 24 hours, starting from 1 to 144, and counting the number of times the WeChat APP is opened in each ten minutes, recorded as count t , where t represents the tth ten minutes.
[0095] Based on the number of times the WeChat APP is opened, calculate the probability within the tth ten minutes:
[0096] Next, the time interval is divided into multiple time periods based on the information entropy of time:
[0097] a. Calculate the total information entropy of the entire time interval: Where T=144.
[0098] b. Assume that the time interval is segmented at the xth point, and the value range of x is [2,143]. Taking the segmentation at the 10th point as an example, the time interval can be divided into two parts: 1 to 10 and 11 to 144. Next, the weighted average information entropy of the two time periods [1,10] and [11,144] after segmentation is calculated:
[0099] c. Calculate the difference between the weighted average information entropy and the total information entropy: ΔE(10)=Ent([1,T])-Ent([1,T];10).
[0100] d. Repeat steps b and c. Since the value range of x in step b is [2,143], each x can get a ΔE(x). Select the largest ΔE(x) and use the corresponding x as the time segmentation point. Assuming that when x is 20, ΔE is the largest, then the time segmentation point is 20, and the time interval is divided into two time segments [1,20] and [21,144].
[0101] After step d, two time periods can be obtained: [1, x] and [x+1, 144]. Repeat steps a to d for the two divided time periods until the number of segments after the time interval is divided reaches a preset threshold. For example, if the preset threshold is 24, the iteration stops when the complete time interval is divided into 24 time periods.
[0102] After the time period is divided, the first panoramic data within the time interval is counted based on the time period to obtain the second panoramic data sequence corresponding to each time period. <t j ,d(t j )> is stored in the MySQL database in the form of. For example, if one of the divided time periods is [1,3], then this time period corresponds to the first half hour of the time interval. Assuming that the acquisition frequency is once per second, the second panoramic data sequence corresponding to this time period corresponds to 1800 second panoramic data.
[0103] It should be noted that the above-mentioned time intervals, time units and other parameter sizes are all examples. In actual applications, they can be set as needed and are not limited to the above-mentioned values.
[0104] Step 104, the step of generating a second feature set according to a preset second feature extraction algorithm and the second panoramic data, includes: extracting time domain features from the second panoramic data sequence corresponding to the time period according to the time domain feature extraction algorithm, and forming the second feature set by the time domain features corresponding to the multiple time periods.
[0105] Since the panoramic data sequence has the characteristics of a time series, the time domain feature extraction algorithm can be used to extract the time domain features from the panoramic data sequence as the second feature. Taking the acceleration sensor data as an example, the acceleration sensor data sequence is statistically analyzed to obtain the peak value, mean value, root mean square value, kurtosis index, waveform factor and other time domain features in the data sequence as the second feature. In this way, the time domain features f(t j ), where j∈(1,m), f(t j ) constitutes the second feature set Y = [f(t1), f(t2), ..., f(t j ),……,f(t m )].
[0106] Step 105: Cross-merge the first feature set and the second feature set to generate a cross-feature set based on geographic location and time information.
[0107] After obtaining the first feature set X and the second feature set Y, the features in the two feature sets are cross-fused.
[0108] Specifically, in another optional implementation, step 105, the step of cross-merging the first feature set and the second feature set to generate a cross feature set based on geographic location and time information, includes:
[0109] A Cartesian product of the first feature set and the second feature set is calculated, and the Cartesian product is used as the intersection feature set.
[0110] In the embodiment of the present application, the features are cross-fused by calculating the Cartesian product of the two sets. Specifically, the Cartesian product of the first feature set X and the second feature set Y can be expressed as X×Y and Y×X. The elements in the Cartesian product are pairs of elements in the first feature set X and the elements in the second feature set Y, as follows:
[0111] X×Y={{f(a1),f(t1)},{f(a1),f(t2)},……,{f(a1),f(t m )},……,
[0112] {f(a2),f(t1)},{f(a2),f(t2)},……,{f(a2),f(tm )},……,
[0113] {f(a n ),f(t1)},{f(a n ),f(t2)},……,{f(a n ),f(t m )}}
[0114] Y×X={{f(t1),f(a1)},{f(t1),f(a2)},……,{f(t1),f(a n )},……,
[0115] {f(t2),f(a1)},{f(t2),f(a2)},……,{f(t2),f(a n )},……,
[0116] {f(t m ),f(a1)},{f(t m ),f(a2)},……,{f(t m ),f(a n )}}
[0117] The cross-fusion features obtained through Cartesian product can reflect both geographic location and time information. For example, {f(a1), f(t1)} represents the panoramic features of the user at the a1 geographic location during the t1 time period. These cross-features constitute a cross-feature set. By constructing features based on geographic location (i.e., the first feature), the mobile properties of panoramic data can be better characterized; by constructing panoramic features based on time (i.e., the second feature), the panoramic preferences and habits of users in the time dimension can be more accurately reflected (panoramic data reflects panoramic information); in addition, by cross-fusion of geographic location-based features and time-based features, on the one hand, the scale and scope of panoramic features are significantly expanded; on the other hand, by simultaneously fusing geographic trajectory attributes and time attributes, in the identification of panoramic categories, the user's behavioral habits and panoramic preferences can be more accurately characterized, greatly improving the recognition accuracy of panoramic categories and the user experience in practical applications.
[0118] In specific implementation, the present application is not limited by the execution order of the various steps described. If no conflict occurs, some steps can be performed in other orders or simultaneously.
[0119] As can be seen from the above, the feature extraction method proposed in the embodiment of the present application collects panoramic data based on geographic location and time information respectively, and for the panoramic data collected based on geographic location, features are extracted therefrom to form a first feature set; at the same time, for the panoramic data collected based on time information, features are extracted therefrom to form a second feature set, and the first feature set and the second feature set are cross-fused to generate a cross feature set based on geographic location and time information, which not only increases the dimension and scale of the features obtained from the panoramic data, but also can fuse geographic location attributes and time attributes, making the features more comprehensive, and in the identification of scenario categories, can better characterize the user's behavioral habits and preferences, thereby improving the accuracy of scenario category identification.
[0120] In one embodiment, a feature extraction device is also provided. Figure 5 , Figure 5 The structural diagram of the feature extraction device 400 provided in the embodiment of the present application. The feature extraction device 400 is applied to an electronic device, and the feature extraction device 400 includes a first data acquisition module 401, a first feature extraction module 402, a second data acquisition module 403, a second feature extraction module 404 and a feature fusion module 405, as follows:
[0121] The first data collection module 401 is used to collect first panoramic data based on the geographical location.
[0122] The first feature extraction module 402 is used to generate a first feature set according to a preset first feature extraction algorithm and the first panoramic data.
[0123] The second data acquisition module 403 is used to acquire second panoramic data based on time information.
[0124] The second feature extraction module 404 is used to generate a second feature set according to a preset second feature extraction algorithm and the second panoramic data.
[0125] The feature fusion module 405 is used to cross-fuse the first feature set and the second feature set to generate a cross feature set based on geographic location and time information.
[0126] In some embodiments, the feature fusion module 405 is further used to: calculate the Cartesian product of the first feature set and the second feature set, and use the Cartesian product as the cross feature set.
[0127] In some embodiments, the first data acquisition module 401 is also used to: collect first panoramic data at a preset frequency and record the geographical location corresponding to each piece of first panoramic data; based on the geographical location, perform statistics on the collected first panoramic data to obtain the first panoramic data corresponding to each geographical location.
[0128] In some embodiments, the first feature extraction module 402 is further used to extract statistical features from the first panoramic data corresponding to each geographical location according to a feature extraction algorithm based on statistics to form the first feature set.
[0129] In some embodiments, the first feature extraction module 402 is further used to: convert the first panoramic data into frequency domain panoramic data according to discrete Fourier transform; extract frequency domain features from the frequency domain panoramic data corresponding to each geographical location to form the first feature set.
[0130] In some embodiments, the first feature extraction module 402 is further used to: count the geographical locations of the first panoramic data and generate a geographical location list;
[0131] Determining a density parameter for cluster analysis, clustering the geographical locations in the geographical location list according to the density parameter and a preset clustering algorithm, and acquiring a plurality of geographical location sets;
[0132] Taking the geographical location set as a reference, performing statistics on the collected first panoramic data to obtain the first panoramic data corresponding to each geographical location set;
[0133] According to a preset first feature extraction algorithm, first features corresponding to the geographical location set are obtained from the corresponding first panoramic data, and the first features corresponding to the plurality of geographical location sets constitute the first feature set.
[0134] In some embodiments, the second data acquisition module 403 is further used to: acquire the second panoramic data within the time interval according to a preset frequency, and record the time information of each piece of the second panoramic data;
[0135] Dividing the time interval into a plurality of time periods;
[0136] Taking the time period as a reference, statistics are collected on the second panoramic data within the time interval to obtain a second panoramic data sequence corresponding to each time period;
[0137] The time domain features are extracted from the second panoramic data sequence corresponding to the time period according to the time domain feature extraction algorithm, and the second feature set is formed by the time domain features corresponding to the multiple time periods.
[0138] In some embodiments, the second data acquisition module 403 is further used to:
[0139] In some embodiments, the total information entropy of the time interval is calculated based on the second panoramic data in the time interval;
[0140] According to a time division point in a preset value range, the time interval is divided into two time periods, and the weighted average information entropy of the two divided time periods is calculated;
[0141] Determine the time segmentation point at which the difference between the weighted average information entropy and the total information entropy is the largest;
[0142] Dividing the time interval into a first time period and a second time period according to the time division point;
[0143] Taking the first time period and the second time period as new time intervals, repeat the step of calculating the total information entropy of the time interval based on the second panoramic data within the time interval to the step of dividing the time interval into the first time period and the second time period according to the time division point, until the number of segments of the time interval is greater than a preset threshold.
[0144] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.
[0145] As can be seen from the above, in the feature extraction device proposed in the embodiment of the present application, the first data acquisition module 401 acquires first panoramic data based on geographic location, and the second data acquisition module 403 acquires second panoramic data based on time information. For the first panoramic data acquired based on geographic location, the first feature extraction module 402 extracts features therefrom to form a first feature set; at the same time, for the second panoramic data acquired based on time information, the second feature extraction module 404 extracts features therefrom to form a second feature set. The feature fusion module 405 cross-fuses the first feature set and the second feature set to generate a cross-feature set based on geographic location and time information, which not only increases the dimension and scale of the features obtained from the panoramic data, but also can fuse geographic location attributes and time attributes, making the features more comprehensive. In the identification of scenario categories, it can better characterize the user's behavioral habits and preferences, thereby improving the accuracy of scenario category identification.
[0146] The present application also provides an electronic device. The electronic device may be a smart phone, a tablet computer, or the like. Figure 6 As shown, Figure 6 The first structural diagram of the electronic device provided in the embodiment of the present application. The electronic device 300 includes a processor 301 and a memory 302. The processor 301 is electrically connected to the memory 302.
[0147] The processor 301 is the control center of the electronic device 300. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or calling computer programs stored in the memory 302, and calling data stored in the memory 302, thereby monitoring the electronic device as a whole.
[0148] In this embodiment, the processor 301 in the electronic device 300 loads instructions corresponding to one or more computer program processes into the memory 302 according to the following steps, and the processor 301 runs the computer program stored in the memory 302 to implement various functions:
[0149] Collecting first panoramic data based on the geographical location;
[0150] Generate a first feature set according to a preset first feature extraction algorithm and the first panoramic data;
[0151] collecting second panoramic data based on time information;
[0152] Generate a second feature set according to a preset second feature extraction algorithm and the second panoramic data;
[0153] The first feature set and the second feature set are cross-fused to generate a cross feature set based on geographic location and time information.
[0154] In some embodiments, when the first feature set and the second feature set are cross-fused to generate a cross feature set based on geographic location and time information, the processor 301 performs the following steps:
[0155] A Cartesian product of the first feature set and the second feature set is calculated, and the Cartesian product is used as the intersection feature set.
[0156] In some embodiments, when collecting the first panoramic data based on the geographical location, the processor 301 performs the following steps:
[0157] Collect panoramic data at a preset frequency and record the geographical location corresponding to each piece of panoramic data;
[0158] Based on the geographical location, the collected first panoramic data are counted to obtain the first panoramic data corresponding to each geographical location.
[0159] In some embodiments, when generating a first feature set according to a preset first feature extraction algorithm and the first panoramic data, the processor 301 performs the following steps:
[0160] According to a feature extraction algorithm based on statistics, statistical features are extracted from the first panoramic data corresponding to each geographical location to form the first feature set.
[0161] In some embodiments, when generating a first feature set according to a preset first feature extraction algorithm and the first panoramic data, the processor 301 performs the following steps:
[0162] Converting the first panoramic data into frequency domain panoramic data according to discrete Fourier transform;
[0163] Frequency domain features are extracted from the frequency domain panoramic data corresponding to each geographical location to form the first feature set.
[0164] In some embodiments, based on the geographical location as a reference, the collected first panoramic data is counted, and when the first panoramic data corresponding to each geographical location is obtained, the processor 301 performs the following steps:
[0165] Counting the geographical locations of the panoramic data and generating a geographical location list;
[0166] Determining a density parameter for cluster analysis, clustering the geographical locations in the geographical location list according to the density parameter and a preset clustering algorithm, and acquiring a plurality of geographical location sets;
[0167] Taking the geographical location set as a benchmark, collecting statistics on the panoramic data, and obtaining the panoramic data corresponding to each geographical location set;
[0168] When generating a first feature set according to a preset first feature extraction algorithm and the first panoramic data, the processor 301 performs the following steps:
[0169] According to a preset first feature extraction algorithm, first features corresponding to the geographical location set are obtained from corresponding panoramic data, and the first features corresponding to the plurality of geographical location sets constitute the first feature set.
[0170] In some embodiments, when collecting the second panoramic data based on the time information, the processor 301 performs the following steps:
[0171] Collect panoramic data within a time interval according to a preset frequency, and record the time information of each piece of panoramic data;
[0172] Dividing the time interval into a plurality of time periods;
[0173] Taking the time period as a benchmark, statistics are collected on the panoramic data within the time interval to obtain a panoramic data sequence corresponding to each time period;
[0174] When generating a second feature set according to a preset second feature extraction algorithm and the second panoramic data, the processor 301 performs the following steps: extracting time domain features from the panoramic data sequence corresponding to the time period according to the time domain feature extraction algorithm, and forming the second feature set by the time domain features corresponding to the multiple time periods.
[0175] In some embodiments, when the time interval is divided into multiple time periods, the processor 301 performs the following steps:
[0176] Calculating the total information entropy of the time interval according to the second panoramic data in the time interval;
[0177] According to a time division point in a preset value range, the time interval is divided into two time periods, and the weighted average information entropy of the two divided time periods is calculated;
[0178] Determine the time segmentation point at which the difference between the weighted average information entropy and the total information entropy is the largest;
[0179] Dividing the time interval into a first time period and a second time period according to the time division point;
[0180] Taking the first time period and the second time period as new time intervals, repeat the step of calculating the total information entropy of the time interval based on the second panoramic data within the time interval to the step of dividing the time interval into the first time period and the second time period according to the time division point, until the number of segments of the time interval is greater than a preset threshold.
[0181] The memory 302 can be used to store computer programs and data. The computer programs stored in the memory 302 contain instructions that can be executed in the processor. The computer programs can form various functional modules. The processor 301 executes various functional applications and feature extraction by calling the computer programs stored in the memory 302.
[0182] In some embodiments, Figure 7 As shown, Figure 7 A second structural diagram of an electronic device provided in an embodiment of the present application. The electronic device 300 further includes: a radio frequency circuit 303, a display screen 304, a control circuit 305, an input unit 306, an audio circuit 307, a sensor 308, and a power supply 309. The processor 301 is electrically connected to the radio frequency circuit 303, the display screen 304, the control circuit 305, the input unit 306, the audio circuit 307, the sensor 308, and the power supply 309, respectively.
[0183] The radio frequency circuit 303 is used to send and receive radio frequency signals to communicate with network devices or other electronic devices through wireless communication.
[0184] The display screen 304 may be used to display information input by a user or information provided to a user and various graphical user interfaces of the electronic device. These graphical user interfaces may be composed of images, texts, icons, videos and any combination thereof.
[0185] The control circuit 305 is electrically connected to the display screen 304 and is used to control the display screen 304 to display information.
[0186] The input unit 306 can be used to receive input numbers, character information or user feature information (such as fingerprints), and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. The input unit 306 can include a fingerprint recognition module.
[0187] The audio circuit 307 can provide an audio interface between the user and the electronic device through a speaker and a microphone. The audio circuit 307 includes a microphone. The microphone is electrically connected to the processor 301. The microphone is used to receive voice information input by the user.
[0188] The sensor 308 is used to collect external environment information. The sensor 308 may include one or more sensors such as an environment brightness sensor, an acceleration sensor, and a gyroscope.
[0189] The power supply 309 is used to supply power to various components of the electronic device 300. In some embodiments, the power supply 309 may be logically connected to the processor 301 through a power management system, so that the power management system can manage charging, discharging, power consumption, and other functions.
[0190] although Figure 7 Not shown, the electronic device 300 may also include a camera, a Bluetooth module, etc., which will not be described in detail here.
[0191] As can be seen from the above, an embodiment of the present application provides an electronic device, which can collect panoramic data based on geographic location and time information respectively, and for the first panoramic data collected based on the geographic location, features are extracted therefrom to form a first feature set; at the same time, for the second panoramic data collected based on the time information, features are extracted therefrom to form a second feature set, and the first feature set and the second feature set are cross-fused to generate a cross-feature set based on geographic location and time information, which not only increases the dimension and scale of the features obtained from the panoramic data, but also can fuse the geographic location attributes and time attributes, making the features more comprehensive, and in the identification of scenario categories, can better characterize the user's behavioral habits and preferences, thereby improving the accuracy of scenario category identification.
[0192] An embodiment of the present application further provides a storage medium, wherein a computer program is stored in the storage medium. When the computer program is executed on a computer, the computer executes the feature extraction method described in any of the above embodiments.
[0193] It should be noted that, a person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the storage medium can include but is not limited to: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0194] The term "module" used herein can be regarded as a software object executed on the computing system. The different components, modules, engines and services described herein can be regarded as implementation objects on the computing system. The apparatus and method described herein can be implemented in software, and of course can also be implemented in hardware, all within the scope of protection of this application.
[0195] The terms "first", "second", and "third" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the listed steps or modules, but some embodiments also include steps or modules that are not listed, or some embodiments also include other steps or modules inherent to these processes, methods, products, or devices.
[0196] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0197] The feature extraction method, device, storage medium and electronic device provided in the embodiments of the present application are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.
Claims
1. A feature extraction method, characterized in that: include: Collecting first panoramic data at a preset frequency, and recording a geographical location corresponding to each piece of first panoramic data; Counting the geographical locations of the first panoramic data and generating a geographical location list; Determining a density parameter for cluster analysis, clustering the geographical locations in the geographical location list according to the density parameter and a preset clustering algorithm, and acquiring a plurality of geographical location sets; Taking the set of geographical locations as a reference, statistics are performed on the collected first panoramic data to obtain first panoramic data corresponding to each geographical location, where the first panoramic data is represented in the form of a third-order tensor; Generate a first feature set according to a preset first feature extraction algorithm and the first panoramic data; collecting second panoramic data based on time information; Generate a second feature set according to a preset second feature extraction algorithm and the second panoramic data; The first feature set and the second feature set are cross-fused to generate a cross feature set based on geographic location and time information.
2. The feature extraction method according to claim 1, further comprising: A Cartesian product of the first feature set and the second feature set is calculated, and the Cartesian product is used as the intersection feature set.
3. The feature extraction method according to claim 1, further comprising: According to a feature extraction algorithm based on statistics, statistical features are extracted from the first panoramic data corresponding to each geographical location to form the first feature set.
4. The feature extraction method according to claim 1, further comprising: Converting the first panoramic data into frequency domain panoramic data according to discrete Fourier transform; Frequency domain features are extracted from the frequency domain panoramic data corresponding to each geographical location to form the first feature set.
5. The feature extraction method according to claim 1, further comprising: According to a preset first feature extraction algorithm, first features corresponding to the geographical location set are obtained from the corresponding first panoramic data, and the first features corresponding to the plurality of geographical location sets constitute the first feature set.
6. The feature extraction method according to claim 1 or 2, characterized in that: The step of collecting the second panoramic data based on the time information includes: Collecting the second panoramic data within the time interval according to a preset frequency, and recording the time information of each piece of the second panoramic data; Dividing the time interval into a plurality of time periods; Taking the time period as a reference, statistics are collected on the second panoramic data within the time interval to obtain a second panoramic data sequence corresponding to each time period; The time domain features are extracted from the second panoramic data sequence corresponding to the time period according to the time domain feature extraction algorithm, and the second feature set is formed by the time domain features corresponding to the multiple time periods.
7. The feature extraction method according to claim 6, characterized in that: The steps of dividing the time interval into multiple time periods include: Calculating the total information entropy of the time interval according to the second panoramic data in the time interval; According to a time division point in a preset value range, the time interval is divided into two time periods, and the weighted average information entropy of the two divided time periods is calculated; Determine the time segmentation point at which the difference between the weighted average information entropy and the total information entropy is the largest; Dividing the time interval into a first time period and a second time period according to the time division point; Taking the first time period and the second time period as new time intervals, repeat the step of calculating the total information entropy of the time interval based on the second panoramic data within the time interval to the step of dividing the time interval into the first time period and the second time period according to the time division point, until the number of segments of the time interval is greater than a preset threshold.
8. A feature extraction device, characterized in that: include: A first data collection module, used to collect first panoramic data at a preset frequency and record the geographical location corresponding to each piece of first panoramic data; Counting the geographical locations of the first panoramic data to generate a geographical location list; determining a density parameter for cluster analysis, clustering the geographical locations in the geographical location list according to the density parameter and a preset clustering algorithm, and acquiring a plurality of geographical location sets; Taking the set of geographical locations as a reference, statistics are performed on the collected first panoramic data to obtain first panoramic data corresponding to each geographical location, where the first panoramic data is represented in the form of a third-order tensor; A first feature extraction module, used for generating a first feature set according to a preset first feature extraction algorithm and the first panoramic data; A second data acquisition module, used for acquiring second panoramic data based on time information; A second feature extraction module, used for generating a second feature set according to a preset second feature extraction algorithm and the second panoramic data; The feature fusion module is used to cross-fuse the first feature set and the second feature set to generate a cross feature set based on geographic location and time information.
9. The feature extraction device according to claim 8, characterized in that: The feature fusion module is further used to: calculate the Cartesian product of the first feature set and the second feature set, and use the Cartesian product as the cross feature set.
10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed on a computer, the computer is enabled to execute the feature extraction method according to any one of claims 1 to 7.
11. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: The processor is used to execute the feature extraction method according to any one of claims 1 to 7 by calling the computer program.
