Video acquisition method

Abstract

The invention discloses a video acquisition method, and relates to the technical field of artificial intelligence. The method comprises the following steps: classifying users to obtain a plurality of user groups; according to historical behavior data of each user in each user group, an AOI corresponding to each user group is obtained, and the historical behavior data comprises historical eye movement data; and according to the AOI corresponding to each user group, dividing a target eye movement video to obtain an AOI video clip corresponding to each user group.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a video acquisition method. Background Art

[0002] Areas of Interest (AOI) are the areas in the stimulus materials where the key elements defined in the eye movement experiment are located. The user's eye movement data in this area is collected. Through AOI analysis, the visualized eye movement data of the user's interest area can be obtained, interference data can be eliminated, and the user's interests, preferences and usage can be understood, so as to improve the application interface. There are two ways to obtain user eye movement data in related technologies. One is to use a wearable eye tracker to obtain user eye movement data, but when the user uses a wearable eye tracker, his focus cannot always be maintained in the target area. The other is to use a remote test eye tracker to obtain user eye movement data, but the aforementioned device needs to greatly limit the user's posture or movement, and cannot obtain the user's most natural state data.

[0003] Therefore, AOI processing of eye movement data can only rely on manual delineation, which is time-consuming and labor-intensive. Summary of the invention

[0004] The embodiment of the present application provides a video acquisition method to solve the problem that AOI processing of eye movement data in the related art can only rely on manual delineation, which is time-consuming and labor-intensive.

[0005] In a first aspect, an embodiment of the present application provides a video acquisition method, comprising:

[0006] Classify users to obtain multiple user groups;

[0007] Obtaining an area of ​​interest AOI corresponding to each user group according to the historical behavior data of each user in each user group, wherein the historical behavior data includes historical eye movement data;

[0008] The target eye movement video is divided according to the AOI corresponding to each of the user groups to obtain the AOI video segments corresponding to each of the user groups.

[0009] In a second aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, which, when executed by a computer, implement the steps of the method described in the first aspect.

[0010] In an embodiment of the present application, users are first classified to obtain multiple user groups, and then the AOI corresponding to each user group is obtained based on the historical behavior data of each user in each user group, the historical behavior data includes historical eye movement data, and finally the target eye movement video is divided according to the AOI corresponding to each user group to obtain the AOI video clips corresponding to each user group. The embodiment of the present application analyzes user portraits, obtains the AOI corresponding to different users according to different user types, and then automatically divides the eye movement video based on the AOI corresponding to different user groups, improves the processing efficiency of the AOI of the eye movement data, and finally obtains the AOI video clips of interest to the user, and divides the video by the AOI corresponding to different user groups, so that the obtained video better meets the needs of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1 is a flow chart of a video acquisition method provided in an embodiment of the present application;

[0013] Figure 2 is a schematic diagram of a valid area in an eye movement video provided in an embodiment of the present application;

[0014] Figure 3 is a schematic diagram of an optimized effective area provided in an embodiment of the present application;

[0015] Figure 4 It is a schematic diagram of distinguishing target areas provided in an embodiment of the present application;

[0016] Figure 5 is a schematic diagram of a video acquisition device provided in an embodiment of the present application;

[0017] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0020] The following is combined with Figures 1 to 6 , a video acquisition method provided in an embodiment of the present application is described in detail through specific embodiments and application scenarios.

[0021] like Figure 1 As shown, it is a flow chart of a video acquisition method provided by an embodiment of the present application. Figure 1 As shown, the video acquisition method may include: contents shown in S101 to S103.

[0022] In S101, users are classified to obtain multiple user groups.

[0023] Among them, the classification of users can be classified according to the functional requirements of users. For example, users can be divided into four groups: basic storage function users, basic viewing function users, special processing function users, and marketing welfare function users.

[0024] Among them, users of basic storage functions will focus on the behavioral information corresponding to the user backup and upload function modules; the user group of basic viewing functions will focus on the information corresponding to the documents, albums and other modules; the user group of special processing functions will focus on functions such as notes and photo recognition; the user group of marketing welfare functions will focus on the relevant indicator information of the membership system.

[0025] It is worth noting that due to the different needs of different users, in the process of obtaining AOI based on different user groups, the feature weight configurations of different groups will be different. For example, for the user group with basic storage functions, their data is more concentrated on indicators such as backup and upload.

[0026] In S102, the area of ​​interest AOI corresponding to each user group is obtained according to the historical behavior data of each user in each user group, where the historical behavior data includes historical eye movement data.

[0027] In this embodiment, since different user groups have different visual focuses, a machine learning algorithm can be used based on the user's historical eye movement data to automatically segment and obtain the visual interest areas of different types of users for specific video content.

[0028] In S103, the target eye movement video is divided according to the AOI corresponding to each user group to obtain the AOI video segments corresponding to each user group.

[0029] In other words, the target eye movement video can be differentiated and divided according to the AOI corresponding to each user group, and the area that different users focus on can be determined, that is, the video slice where the sight enters the target area and leaves the target area, and finally the AOI video clip corresponding to each user group can be obtained. In other words, different simplified videos of the same video can be determined for different user groups to meet user needs.

[0030] In an embodiment of the present application, users are first classified to obtain multiple user groups, and then the AOI corresponding to each user group is obtained based on the historical behavior data of each user in each user group, the historical behavior data includes historical eye movement data, and finally the target eye movement video is divided according to the AOI corresponding to each user group to obtain the AOI video clips corresponding to each user group. The embodiment of the present application analyzes user portraits, obtains the AOI corresponding to different users according to different user types, and then automatically divides the eye movement video based on the AOI corresponding to different user groups, improves the processing efficiency of the AOI of the eye movement data, and finally obtains the AOI video clips of interest to the user, and divides the video by the AOI corresponding to different user groups, so that the obtained video better meets the needs of the user.

[0031] In a possible implementation of the present application, the video acquisition method may also include: when a user uses an eye tracker, acquiring status data of the eye tracker and the terminal device; determining a target area where the terminal device is located in the video data based on video data of the eye movement data acquired by the eye tracker; and acquiring eye movement data of the user in the target area.

[0032] In this embodiment, the state data of the eye tracker and the terminal device can be determined based on the user using the eye tracker to view the terminal device. That is to say, the wearable eye tracker can be used to study the scene of a single terminal device (which can be a mobile device), so that the user's eye movement data obtained is the user's natural state eye movement data, which contains the user's variable, random task and non-task eye movement data. The most common problems in the study of terminal device scenes are the occlusion of the target area by hand operation, and the situation where the device is not facing or perpendicular to the user's line of sight. The target area where the terminal is located in the video can be extracted first, such as Figure 2 As shown, the terminal device may have a certain inclination relative to the eye tracker at this time, and can be optimized and adjusted to obtain a planar viewing angle, such as Figure 3 shown.

[0033] It can record the screen changes of the terminal device on the timeline, the device gravity sensor data, etc., and combine the semi-automatic tracking method to quickly process and delineate the position of the target area in the original video of the eye tracker on the timeline, and establish the coordinate range of the overall AOI area, the target AOI area and the shallow AOI area, and divide the eye movement data inside and outside the target area. Combined with the records of the special screen recording software, it matches the user's screen switching on the corresponding timeline and maps the eye movement data of the corresponding interface by establishing a coordinate system, so as to quickly obtain the user's corresponding eye movement data in each interface of the terminal device.

[0034] In a possible implementation of the present application, obtaining status data of an eye tracker and a terminal device may include: collecting first acceleration data of the eye tracker and second acceleration data of the terminal device; filtering a sequence of the first acceleration data and the second acceleration data based on a median filter to obtain smoothed first acceleration data and second acceleration data, wherein a time window of the median filter is predicted using a standard deviation dynamic adjustment model; and obtaining a tilt angle sequence of the eye tracker and a tilt angle sequence of the terminal device based on the smoothed first acceleration data and the second acceleration data.

[0035] In this embodiment, the data of the eye tracker and the terminal device can be collected synchronously, that is, they can be started and stopped synchronously, and the screen switching, dwell time, and specific interaction of the target interactive controls in the interface performed by the tester on the corresponding terminal device can be recorded. At the same time, the gravity sensor data of the mobile device can be recorded. The gravity sensor information of the device can be calibrated and adjusted first, for example, it can be adjusted through 8-character calibration or other methods, which is subject to the actual application and is not limited in this embodiment.

[0036] Specifically, the acceleration of the x, y, and z axes of the terminal device (V x 、V y 、V z ), the eye tracker records the acceleration of the tester's head on the x, y, and z axes (A X , A Y , A Z ), the acceleration data sequence is filtered using median filtering to generate a smoothed (v x 、v y 、v z ) and (a X 、a Y 、a Z ), the time window of the median filter uses the standard deviation to dynamically adjust the model prediction window size. The standard deviation calculation formula is as follows:

[0037]

[0038] When StdDev>Q, Wupdate = W initial × 120%, W initial = W update

[0039] When StdDev < Q, W update = W initial × 80%, W initial = W update

[0040] where StdDev i is the standard deviation of the data points within the window calculated at the data point x i , x j is the value of each data point within the window, and the initial window W initial is set to a size of 30. The adjustment rule is: when StdDev i the standard deviation is greater than the fixed threshold Q, increase the window length by the first preset value, such as increasing by 20%, and when it is less than the threshold, decrease the window length by the first preset value. The Q value can be set to 3.5 or other values, subject to actual application. The size of the first preset value can also be adjusted according to actual application, and no specific limitations are made in this embodiment. Update the window size to W update , and use the updated window size for median filtering processing at the next data point.

[0041] Calculate the tilt angle from the acceleration to determine the change in the posture of the mobile device. The formula is as follows. Here, only the case of the accelerator of the mobile device is used as an example, and the angle of the eye tracker is the same:

[0042] Tilt angle of the x-axis:

[0043] Tilt angle of the y-axis:

[0044] Tilt angle of the z-axis:

[0045] The set is the set of angle states of the time series. The tilt angle sequence of the terminal device is P1, and the tilt angle sequence of the eye tracker is P2:

[0046] P1 = [i1(θ x , θ y , θ z ), i2(θ x , θ y , θ z ), i3(θ x , θ y , θ z ), i4(θ x , θ y , θ z)...i i (θ x ,θ y ,θ z )]

[0047] P2=[i1(θ x ,θ y ,θ z ),i2(θ x ,θ y ,θ z ),i3(θ x ,θ y ,θ z ),i4(θ x ,θ y ,θ z )...i i (θ x ,θ y ,θ z )].

[0048] In one example, determining a target area where a terminal device is located in the video data based on video data of eye movement data acquired by an eye tracker may include: using the acquired video data of eye movement data to frame a target mobile device area in the eye tracker data video.

[0049] Record the initial position of the target area, where x, y are the x and y positions on the plane coordinates, and w, h are the width and height of the selected area, which also serve as the structural elements for subsequent morphological processing.

[0050] F=(x,y,w,h)

[0051] In a possible implementation of the present application, after determining the target area where the terminal device is located in the video data based on the video data of the eye movement data acquired by the eye tracker, the video acquisition method may further include: based on the video data, converting each frame in the video data into a grayscale frame; binarizing the grayscale image to obtain a high-contrast image; and performing morphological processing on the high-contrast image to obtain processed video data.

[0052] In this embodiment, each frame of the video is read in a loop, the color frame is converted into a grayscale frame by a linear weighted average formula, and then the grayscale image is binarized to obtain a high-contrast image. In order to obtain the most complete target image processing result possible, the high-contrast image can be morphologically processed to obtain processed video data.

[0053] The specific process of morphological processing is to perform a closing operation on the pixels and structural elements of the video image, and to generate a new image by reassigning new pixel values ​​through comparison and overlap processing. The image is first dilated and then eroded to obtain a video image after morphological processing.

[0054] In a possible implementation of the present application, obtaining the user's eye movement data in a target area may include: performing edge detection on the target area to obtain a target area and a background area, where the number of background areas is multiple; using the target area as a positive sample and the background area as a negative sample, inputting them into an initial image area model for training to obtain a target image area model, where the target image area model is used to distinguish the type of image area and determine the bounding box of the target area, and the types include the target area and the background area; inputting each frame of data in the video data into the target image area model for processing to determine the target area in the video data; and deleting the data outside the target area to obtain the eye movement data in the target area.

[0055] In this embodiment, a target tracking algorithm is used to perform edge detection on the target area, distinguish the target from the background, sample multiple image areas around the target frame in the current start frame, and add the image areas containing the framed object as positive samples and the image areas not containing the framed object as negative samples to the target model. The formula is as follows:

[0056] x i =sample(I,u)

[0057] Where I is the input frame, u is the bounding box of the target model, and x i is the sampled image area; then the classifier is established and trained, the formula is as follows:

[0058]

[0059] Among them, ω and b are the parameters of Support Vector Machine (SVM), ξ i is a slack variable, which indicates the size of the error, y i is the image area x i The label of (positive sample is 1, negative sample is -1), C is the regularization parameter, is a constraint condition.

[0060] In the subsequent frame, the model is updated according to the output of the classifier, and the formula is as follows:

[0061] u=update(u,x i ,y i )

[0062] Where u is the bounding box of the target model, xi and i are the sampled image regions and labels. The goal of the update is to enable the model to better match the target object while suppressing background noise.

[0063] The entire video is processed to complete the tracking and marking results of the target area in the video, and the F value of the framed area on each frame is obtained from the data; then, the inappropriate time frame is adjusted again through inspection and adjustment to obtain the final F value. According to the obtained F value, the eye movement data outside the F area is deleted to obtain the eye movement data in the AOI area. It is marked as outside the target area on the timeline to intuitively indicate that the user's line of sight is outside the target area during the entire task time.

[0064] In a possible implementation of the present application, the video acquisition method may further include: adjusting the angle of the target area so that the angle of the target area is aligned with the angle of the eye tracker; establishing a coordinate system based on the target area, determining the coordinates of the target area and the interface range of the target area.

[0065] The P1 and P2 obtained by the above embodiment can know the changes in the posture angles of the two devices in the same time series, and use the coincidence of the +z axis of the two devices and the +x axis and +y axis to keep them on the same plane to achieve the front and parallel orientation of the devices, so as to establish more accurate raw data for subsequent mapping. First, perform the rotation matrix conversion:

[0066] Terminal equipment:

[0067]

[0068] Eye Tracker:

[0069]

[0070] Calculate the angle Δθ that the image of the mobile device needs to be adjusted:

[0071]

[0072] Δθ=R2×P2-R1×P1

[0073] According to the new angle difference, a new angle value is obtained to adjust the angle of the processed target area θx_new=θx+Δθx

[0074] θy_new=θy+Δθy

[0075] θz_new=θz+Δθz

[0076] Update to the new angle value, that is:

[0077] P new=[i1(θx_new,θy_new,θz_new),i2(θx_new,θy_new,θz_new),...,

[0078] i i (θx_new,θy_new,θz_new)]

[0079] By traversing the entire timeline, we can see that the recognition area is always facing the tester, and the three axes are always aligned with the three axes of the eye tracker to ensure that there is no image tilt or distortion, which solves the problems of device tilt, not parallel to the front, and hand occlusion during user use. This lays the foundation for the subsequent establishment of accurate mapping.

[0080] In a possible implementation of the present application, the video acquisition method may further include: obtaining interface images of the target AOI and shallow AOI corresponding to the eye movement data by matching the timeline of the video data based on the coordinates of the target area and the interface range of the target area.

[0081] In this embodiment, the lower left corner of the rectangular AOI area can be used as the origin (0,0) to establish an affine transformation matrix, and the coordinates of the eye movement points in the AOI area are recalculated as new pixel coordinates (x1, y1) with the lower left corner as the origin (0,0), and the coordinates of the points in the original interface image are (x2, y2). The length and width of the interface image are recorded as (w1, h1), and bilinear interpolation is used to calculate the eye movement position on the actual interface image. The formula is as follows:

[0082] x2=a×x+(1-a)×(x1+1)

[0083] y2=b×y+(1-b)×(y1+1)

[0084] Where a and b are the magnified ratios of the interface image (w1, h1) corresponding to the AOI area (w, h). Through the matching of the timeline, a more high-definition and non-interference interface image of the target AOI area and the shallow AOI area with eye movement data is obtained.

[0085] In a possible implementation of the present application, the area of ​​interest AOI corresponding to each user group is obtained based on the historical behavior data of each user in each user group, which can include: determining the interest of each user group in different modules in the terminal device based on the click behavior, eye movement data, usage preferences and terminal device data of each user in each user group in the terminal device; based on the interest, determining the target AOI and shallow AOI corresponding to different user groups.

[0086] In other words, based on the behavioral data of different user groups, such as the click behavior, eye movement data, usage preferences, terminal devices and other data of each user group in the terminal device, the interest level of each user group in different modules in the terminal can be determined to adapt to the visual focus of different groups and obtain the target AOI area and shallow AOI area corresponding to different user groups.

[0087] Specifically, the target area can be divided into target AOI and shallow AOI through binary classification, such as Figure 4 As shown, the black frame is the target AOI, and the remaining part is the shallow AOI.

[0088] In a possible implementation of the present application, the target eye movement video is divided according to the AOI corresponding to each user group to obtain the AOI video clips corresponding to each user group, which may include: adding user type labels to the eye movement video data of different users according to the type of user group; when the user uses an eye tracker, by capturing the user's gaze trajectory and gaze point distribution, determining whether the user's line of sight enters the target AOI and the length of time the user stays in the target AOI; obtaining slices of the eye movement video whose length of stay in the target AOI exceeds a length threshold; and splicing the slices to obtain the AOI video clips corresponding to the user groups.

[0089] In this embodiment, user group identifications can be added to the eye movement video data of different users, such as basic storage function user identification, basic viewing function user identification, special processing function user identification, and marketing welfare function user identification, so as to distinguish different user groups and lay the foundation for subsequent personalized video processing. In the process of the user using the eye tracker, the user's gaze trajectory and gaze point distribution are captured in real time to determine whether the user's sight enters the predefined target AOI area and the length of time the user stays in the AOI. By setting the AOI attention time threshold for different groups, if the sight stays for more than the threshold time, it is determined that the user has a strong visual attention to the AOI area, and the eye movement video of the time period is sliced. When the user's sight enters and leaves the AOI area, the specific time period information is recorded, and the video clips of the user's deep attention are extracted with the identified AOI area as the center. For the shallow attention of users in the shallow AOI area, the time period is also recorded as the plot and situation background information to provide context for the main clips. Then, according to different user groups and corresponding target AOI areas, the video clips are automatically filtered and classified, the clips that users pay close attention to are retained, and other clips are evaluated to determine whether to retain or filter, so as to generate a focused video summary result. Finally, the selected video clips are post-produced by smooth transition processing, interpolation technology, etc., so that each clip is naturally spliced ​​together to generate a coherent personalized video summary. Repeating the entire process can customize personalized and simplified video results for the same video for different user groups.

[0090] In a possible implementation of the present application, users are classified to obtain multiple user group types, which may include: obtaining multiple user behavior categories based on the user's historical behavior data; based on the multiple user behavior categories, users are divided into multiple user groups, including a basic storage function user group, a basic viewing function user group, a special processing function user group, and a marketing welfare function user group.

[0091] That is to say, users can be classified according to their functional needs. For example, users can be divided into four groups: basic storage function users, basic viewing function users, special processing function users, and marketing welfare function users.

[0092] Among them, users of basic storage functions will focus on the behavioral information corresponding to the user backup and upload function modules; the user group of basic viewing functions will focus on the information corresponding to the documents, albums and other modules; the user group of special processing functions will focus on functions such as notes and photo recognition; the user group of marketing welfare functions will focus on the relevant indicator information of the membership system.

[0093] In the embodiments of the present application, eye movement videos can be processed to remove invalid areas, and then video clips of target AOI areas and shallow AOI areas can be divided according to different user types. The method of automatically dividing eye movement videos can be combined with user portrait classification to solve the current problem of numerous manual correction operations such as non-AOI area cleaning and video picture correction. The process cannot be well sorted out to restore the complete operation link and thinking details of users using and experiencing software products, and the eye movement experiment results are distorted and the data results have limitations.

[0094] like Figure 5 As shown in FIG. 1 , a schematic diagram of a video acquisition device provided in an embodiment of the present application is shown. Figure 5 As shown, the video acquisition device may include: a classification module 501 , a determination module 502 and an acquisition module 503 .

[0095] Among them, the classification module 501 is used to classify users to obtain multiple user groups; the determination module 502 is used to obtain the area of ​​interest AOI corresponding to each user group based on the historical behavior data of each user in each user group, and the historical behavior data includes historical eye movement data; the acquisition module 503 is used to divide the target eye movement video according to the AOI corresponding to each user group, and obtain the AOI video clip corresponding to each user group.

[0096] In the embodiment of the present application, first, the classification module 501 classifies the users to obtain multiple user groups, and then the determination module 502 obtains the area of ​​interest AOI corresponding to each user group based on the historical behavior data of each user in each user group, and the historical behavior data includes historical eye movement data. Finally, the acquisition module 503 divides the target eye movement video according to the AOI corresponding to each user group to obtain the AOI video clips corresponding to each user group. The embodiment of the present application analyzes the user portrait, obtains the AOI corresponding to different users according to different user types, and then automatically divides the eye movement video based on the AOI corresponding to different user groups, improves the processing efficiency of the AOI of the eye movement data, and finally obtains the AOI video clips of interest to the user, and divides the video by the AOI corresponding to different user groups, so that the obtained video better meets the needs of the user.

[0097] In a possible implementation manner of the present application, the video acquisition device may further include: a second acquisition module, a second determination module and a third acquisition module.

[0098] Among them, the second acquisition module is used to obtain the status data of the eye tracker and the terminal device when the user uses the eye tracker; the second determination module is used to determine the target area where the terminal device is located in the video data based on the video data of the eye movement data obtained by the eye tracker; the third acquisition module is used to obtain the eye movement data of the user in the target area.

[0099] In a possible implementation of the present application, the second acquisition module is used to: collect first acceleration data of the eye tracker and second acceleration data of the terminal device; filter the sequence of the first acceleration data and the second acceleration data based on median filtering to obtain smoothed first acceleration data and second acceleration data, wherein the time window of the median filter is predicted using a standard deviation dynamic adjustment model; based on the smoothed first acceleration data and the second acceleration data, obtain the tilt angle sequence of the eye tracker and the tilt angle sequence of the terminal device.

[0100] In a possible implementation manner of the present application, the video acquisition device may further include: a first processing module.

[0101] Among them, the first processing module is used to convert each frame in the video data into a grayscale frame based on the video data; binarize the grayscale image to obtain a high-contrast image; and perform morphological processing on the high-contrast image to obtain processed video data.

[0102] In a possible implementation of the present application, a third acquisition module is used to: perform edge detection on the target area to obtain a target area and a background area, where the number of background areas is multiple; input the target area as a positive sample and the background area as a negative sample into the initial image area model for training to obtain a target image area model, where the target image area model is used to distinguish the type of image area and determine the bounding box of the target area, and the types include the target area and the background area; input each frame of data in the video data into the target image area model for processing to determine the target area in the video data; delete the data outside the target area to obtain eye movement data within the target area.

[0103] In a possible implementation manner of the present application, the video acquisition device may further include: a second processing module.

[0104] The second processing module is used to adjust the angle of the target area so that the angle of the target area is aligned with the angle of the eye tracker; establish a coordinate system based on the target area, determine the coordinates of the target area and the interface range of the target area.

[0105] In a possible implementation manner of the present application, the video acquisition device may further include: a matching module.

[0106] Among them, the matching module is used to obtain the interface images of the target AOI and the shallow AOI corresponding to the eye movement data through matching the time line of the video data based on the coordinates of the target area and the interface range of the target area.

[0107] In a possible implementation of the present application, a determination module 502 is used to determine the interests of each user group in different modules within the terminal device based on the click behavior, eye movement data, usage preferences and terminal device data of each user in the terminal device; based on the interests, determine the target AOI and shallow AOI corresponding to different user groups.

[0108] In a possible implementation of the present application, the acquisition module 503 is used to add user type labels to the eye movement video data of different users according to the type of user group; when the user uses an eye tracker, by capturing the user's gaze trajectory and gaze point distribution, determine whether the user's line of sight enters the target AOI and the length of time the user stays in the target AOI; obtain slices of the eye movement video in which the length of time spent in the target AOI exceeds a duration threshold; and splice the slices to obtain AOI video clips corresponding to the user group.

[0109] In a possible implementation of the present application, the classification module 501 is used to obtain multiple user behavior categories based on the user's historical behavior data; based on the multiple user behavior categories, the users are divided into multiple user groups, and the user groups include a basic storage function user group, a basic viewing function user group, a special processing function user group, and a marketing welfare function user group.

[0110] The function of the video acquisition device of the present application is Figures 1 to 4 The method embodiment shown is described in detail, so for any details not provided in the description of this embodiment, please refer to the relevant descriptions in the aforementioned embodiments, which will not be repeated here.

[0111] like Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601, a memory 602, and a program or instruction stored in the memory 602 and executable on the processor 601. When the program or instruction is executed by the processor 601, each process of the above-mentioned video acquisition processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0112] Optionally, the embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned video acquisition method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0113] Optionally, an embodiment of the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the various processes of the above-mentioned video acquisition method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0114] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0116] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video acquisition method, characterized in that: include: Classify users to obtain multiple user groups; Obtaining an area of ​​interest AOI corresponding to each user group according to the historical behavior data of each user in each user group, wherein the historical behavior data includes historical eye movement data; The target eye movement video is divided according to the AOI corresponding to each of the user groups to obtain the AOI video segments corresponding to each of the user groups.

2. The method according to claim 1, characterized in that The method further comprises: When the user uses an eye tracker, obtaining status data of the eye tracker and the terminal device; Based on the video data of the eye movement data acquired by the eye tracker, determining a target area where the terminal device is located in the video data; Acquire eye movement data of the user in the target area.

3. The method according to claim 2, characterized in that The obtaining of the status data of the eye tracker and the terminal device includes: Collecting first acceleration data of the eye tracker and second acceleration data of the terminal device; Performing filtering processing on the sequence of the first acceleration data and the second acceleration data based on median filtering to obtain smoothed first acceleration data and second acceleration data, wherein the time window of the median filtering is predicted by a standard deviation dynamic adjustment model; Based on the smoothed first acceleration data and the second acceleration data, a tilt angle sequence of the eye tracker and a tilt angle sequence of the terminal device are obtained.

4. The method according to claim 2, characterized in that: After determining the target area where the terminal device is located in the video data based on the eye movement data obtained by the eye tracker, the method further includes: Based on the video data, convert each frame in the video data into a grayscale frame; Binarize the grayscale image to obtain a high-contrast image; Morphological processing is performed on the high contrast image to obtain processed video data.

5. The method according to claim 2, characterized in that: The acquiring the eye movement data of the user in the target area includes: Performing edge detection on the target area to obtain the target area and background areas, wherein the number of the background areas is multiple; The target area is used as a positive sample and the background area is used as a negative sample, and the samples are input into the initial image area model for training to obtain a target image area model, wherein the target image area model is used to distinguish the types of image areas and determine the bounding box of the target area, wherein the types include the target area and the background area; Inputting each frame of data in the video data into the target image region model for processing to determine the target region in the video data; The data outside the target area is deleted to obtain the eye movement data within the target area.

6. The method according to claim 5, characterized in that The method further comprises: Adjusting the angle of the target area so that the angle of the target area is aligned with the angle of the eye tracker; A coordinate system based on the target area is established to determine the coordinates of the target area and the interface range of the target area.

7. The method according to claim 6, characterized in that The method further comprises: Based on the coordinates of the target area and the interface range of the target area, the interface images of the target AOI and the shallow AOI corresponding to the eye movement data are obtained by matching the time line of the video data.

8. The method according to claim 1, characterized in that The step of obtaining the area of ​​interest (AOI) corresponding to each user group according to the historical behavior data of each user in each user group includes: Determine the interests of each user group in different modules in the terminal device based on the click behavior, eye movement data, usage preferences and data of the terminal device of each user in each user group; Based on the interests, target AOIs and shallow AOIs corresponding to different user groups are determined.

9. The method according to claim 1, characterized in that: The target eye movement video is divided according to the AOI corresponding to each user group to obtain the AOI video segments corresponding to each user group, including: According to the type of user group, add user type labels to the eye movement video data of different users; When the user uses an eye tracker, by capturing the gaze trajectory and gaze point distribution of the user, it is determined whether the user's sight enters the target AOI and the length of time the user stays in the target AOI; Obtaining a slice of the eye movement video in which the duration of staying in the target AOI exceeds a duration threshold; The slices are spliced ​​to obtain AOI video clips corresponding to the user group.

10. The method according to claim 1, characterized in that The user classification is performed to obtain multiple user group types, including: Based on the user's historical behavior data, multiple user behavior categories are obtained; Based on the multiple user behavior categories, the users are divided into multiple user groups, including a basic storage function user group, a basic viewing function user group, a special processing function user group, and a marketing welfare function user group.

Images