Information generation method and apparatus

Abstract

The application discloses an information generation method and device, and relates to the technical field of deep learning. A specific embodiment of the method comprises: acquiring a change range of a focusing area of a user's line of sight on a smart display device in a preset time period; in response to determining that the change range meets a preset condition, acquiring pupil diameter change data of the user in the preset time period; and based on the pupil diameter change data, determining emotional information of the user on the focusing area. This embodiment effectively improves the accuracy of the emotional information generated for the focusing area.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to the technical field of deep learning, and especially to an information generation method and device. BACKGROUND

[0002] With the development of metaverse related technologies, more and more users use metaverse smart devices. How to accurately perceive the emotional state of users to virtual space people and things, and obtain the interests and psychological state of people is an important technology for metaverse. Domestic and foreign researches usually take the measurement value of the pupil size of the observed person as an objective index for evaluating the emotional state.

[0003] In the prior art, the pupil size at a single point in time is usually used to determine the user's emotion, but this method is easily disturbed by the environment and lacks an effective method for accurately identifying target objects or target regions. SUMMARY

[0004] Embodiments of the present application provide an information generation method, device, equipment and storage medium.

[0005] According to a first aspect, embodiments of the present application provide an information generation method, which comprises: acquiring a change range of a focus area of a user's line of sight on a smart display device within a preset time period; in response to determining that the change range meets a preset condition, acquiring pupil diameter change data of the user within the preset time period; and determining emotional information of the user to the focus area based on the pupil diameter change data.

[0006] According to a second aspect, embodiments of the present application provide an information generation device, which comprises: a first acquisition module configured to acquire a change range of a focus area of a user's line of sight on a smart display device within a preset time period; a second acquisition module configured to acquire pupil diameter change data of the user within the preset time period in response to determining that the change range meets a preset condition; and a determination emotion module configured to determine emotional information of the user to the focus area based on the pupil diameter change data.

[0007] According to a third aspect, embodiments of the present application provide a smart glasses, which comprises one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the information generation method of any one of the embodiments of the first aspect.

[0008] According to a fourth aspect, embodiments of the present application provide a computer readable medium having a computer program stored thereon, which is executed by a processor to implement the information generation method of any one of the embodiments of the first aspect.

[0009] The application obtains the change range of the focusing area of the user's line of sight on the intelligent display device in a preset time period, obtains pupil diameter change data of the user in the preset time period in response to determining that the change range meets a preset condition, and determines the emotional information of the user on the focusing area based on the pupil diameter change data, thereby overcoming the problems in the prior art that the user emotion is easily disturbed by the environment according to the pupil size at a single point in time, and the focusing area cannot be accurately identified, and effectively improving the accuracy of the emotional information for the focusing area.

[0010] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is an exemplary system architecture diagram to which the present application can be applied;

[0012] Figure 2 is a flowchart of one embodiment of the information generation method according to the present application;

[0013] Figure 3 is a schematic diagram of one application scenario of the information generation method according to the present application;

[0014] Figure 4 is a schematic diagram of another application scenario of the information generation method according to the present application;

[0015] Figure 5 is a schematic diagram of another application scenario of the information generation method according to the present application

[0016] Figure 6 is a flowchart of another embodiment of the information generation method according to the present application;

[0017] Figure 7 is a schematic diagram of one embodiment of the information generation apparatus according to the present application;

[0018] Figure 8 is a structural schematic diagram of a computer system of a server suitable for implementing the embodiments of the present application. DETAILED DESCRIPTION

[0019] Exemplary embodiments of the present application are described below with reference to the accompanying drawings, which include various details of the embodiments of the present application to assist in understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, in order to be clear and concise, descriptions of well-known functions and structures are omitted in the following description.

[0020] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0021] Figure 1 An exemplary system architecture 100 to which the embodiments of the information generation method of the present application can be applied is shown.

[0022] As shown in Figure 1 The system architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium to provide a communication link between the terminal devices 101, 102, 103 and the server 105, between the terminal devices. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0023] A user can use the terminal devices 101, 102, 103 to interact with other terminal devices or the server 105 through the network 104 to receive or send messages, etc. The terminal devices 101, 102, 103 can be installed with client application software, such as video playing application software, communication application software, etc.

[0024] The terminal devices 101, 102, 103 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices, including but not limited to smartphones, smart bracelets, smart glasses (glasses with positioning devices and image acquisition devices, AR glasses, etc.), VR headsets, tablet computers, laptop computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the above-mentioned electronic devices. They can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made herein.

[0025] The server 105 can be a server providing various services, such as obtaining the change range of the user's focus area on the smart display device within a preset time period; in response to determining that the change range meets the preset condition, obtaining the pupil diameter change data of the user within the preset time period; based on the pupil diameter change data, determining the emotional information of the user on the focus area.

[0026] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (such as for providing information generation services), or as a single software or software module. No specific limitation is made herein.

[0027] It should be noted that the information generation method provided in the embodiments of this disclosure can be executed by server 105, terminal devices 101, 102, and 103, or by server 105 and terminal devices 101, 102, and 103 in cooperation with each other. Accordingly, all parts (e.g., units, subunits, modules, and submodules) of the information generation device can be entirely located in server 105, entirely located in terminal devices 101, 102, and 103, or separately located in server 105 and terminal devices 101, 102, and 103.

[0028] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0029] Figure 2 A flowchart 200 illustrates an embodiment of the information generation method applicable to this application. The information generation method includes the following steps:

[0030] Step 201: Obtain the range of change in the focused area of ​​the user's gaze on the smart display device within a preset time period.

[0031] In this embodiment, the execution entity (e.g., Figure 1 The server 105 or terminal devices 101, 102, 103 can determine the focal area of ​​the user's line of sight on the smart display device via a positioning device worn on the user's head, a positioning device set on the smart display device, an optical axis, and a visual axis, and obtain the range of change of the focal area of ​​the user's line of sight on the smart display device within a preset time period.

[0032] The positioning device can be a device for determining location using existing or future technologies, such as UWB (Ultra Wide Band) systems, LANDMARK systems, etc.

[0033] Here, UWB (Ultra Wide Band) technology is a wireless carrier communication technology that uses nanosecond-level non-sinusoidal narrow pulses to transmit data, thus occupying a wide spectrum. UWB technology has advantages such as low transmitted signal power spectral density, insensitivity to channel fading, and high positioning accuracy, making it particularly suitable for high-precision indoor positioning with an error range in the centimeter range.

[0034] Specifically, such as Figure 3As shown, the positioning device worn on the head of the user is a smart glasses with a built-in UWB chip A, and the positioning device arranged on the smart display device is a device with a built-in UWB chip B, and the execution subject can first determine the distance and angle between the smart glasses and the smart display device according to the coordinate P(x, y, z) indicated by the UWB chip A in the spatial rectangular coordinate system, i.e. the first coordinate, and the coordinate O(0, 0, 0) indicated by the UWB chip B, i.e. the second coordinate. The angle between OP and OY can be calculated by the following formula:

[0035]

[0036] In addition, the distance between the smart glasses and the smart display device can also be determined by the time delay of sending data packets between the smart display device and the smart glasses, such as: the time of the smart display device sending data packets is T1, the time of the smart glasses receiving data packets is T2, the time of the smart glasses sending response data packets is T3, and the time of the smart display device receiving response data packets is T4, the distance D between the smart glasses and the smart display device can be represented by the following formula:

[0037]

[0038] Wherein, C is the wave speed.

[0039] In addition, the coordinate difference between the pupil coordinate Q and the coordinate P of the UWB chip A is (dx, dy, dz), the angle between the line connecting the UWB chip A and the UWB chip B and the X axis is β, and the line is parallel to the optical axis.

[0040] Further, the execution subject can calculate the distance of MN through the P, Q coordinates, α, β, θ angles, and the distance D of the line connecting the UWB chip A and the UWB chip B, and finally determine the line of sight focusing area as a square coverage area with the N point coordinate as the center and the MN*2 length as the side length.

[0041] Wherein, α is the Alpha angle, i.e. the angle between the visual axis and the optical axis, and the normal value of α is 4°≤α≤8°.

[0042] Here, the visual axis is the line connecting the light source and the center of the macula of the retina, and the optical axis is an imaginary line connecting the light source and all images.

[0043] It should be pointed out that the visual axis angle can only include the normal visual axis angle, or both the normal visual axis angle and the oblique visual axis angle, which is not limited in the present application.

[0044] In some alternatives, the focusing area is determined by: determining the optical axis based on the first coordinate, the second coordinate, and a preset coordinate difference between the user's pupil coordinate and the first coordinate; determining the center of the focusing area according to a first intersection point of the optical axis and the smart display device; determining a second intersection point of the visual axis and the smart display device according to the optical axis and a preset visual axis angle, and determining a half side length of the focusing area as a distance between the first intersection point and the second intersection point; and determining the focusing area according to the side length and the center of the focusing area.

[0045] In the present implementation, the execution subject can first determine the first coordinate according to the positioning device worn on the user's eye, determine the second coordinate according to the positioning device arranged on the smart display device, and determine the user's eye coordinate according to the first coordinate and a preset coordinate difference between the user's pupil coordinate and the first coordinate, and then determine the optical axis as a line passing through the user's pupil coordinate and parallel to the line connecting the first coordinate and the second coordinate, and determine the center of the focusing area as a first intersection point of the optical axis and the smart display device.

[0046] Further, the second intersection point of the visual axis and the smart display device is determined according to the optical axis and a preset visual axis angle, and a half side length of the focusing area is determined as a distance between the first intersection point and the second intersection point. Further, the focusing area is determined according to the side length and the center of the focusing area.

[0047] The visual axis angle can include a direct visual axis angle (angle between the direct visual axis and the optical axis) and an oblique visual axis angle (angle between the oblique visual axis and the direct visual axis).

[0048] Here, since the human eye visual angle is about 50 degrees in general cases, which refers to the visual range of the direct visual axis, the 50mm standard lens of the camera is closest to the human eye visual angle. In the case of not rotating the eyeball, the maximum visual angle of the two eyes is 124 degrees, and when concentrating attention, it is about one-fifth, i.e., 25 degrees. Concentrating attention is similar to focusing on the central area of the camera, which is close to single-point focusing. In order to more accurately consider the case of oblique vision or eyeball rotation, the direct visual axis angle and the oblique visual axis angle γ can be estimated according to the pupil photo, and the model is trained by marking the sample pupil angle. In general cases, the value range of γ is <25 degrees.

[0049] Specifically, as shown in Figure 4 the direct visual axis angle is α, the oblique visual axis angle is γ, and the human eye focusing area is an area with N as the center point and 2*NR as the side length.

[0050] The implementation determines the optical axis based on the first coordinate, the second coordinate, and the coordinate difference value of the preset user pupil coordinate and the first coordinate, determines the center of the focus area according to the first intersection point of the optical axis and the smart display device, determines the second intersection point of the visual axis and the smart display device according to the optical axis and the preset visual axis angle, and determines the distance between the first intersection point and the second intersection point as the half side length of the focus area, thereby improving the accuracy of the determined focus area.

[0051] In step 202, in response to determining that the change range meets the preset condition, the pupil diameter change data of the user in the preset time period is acquired.

[0052] In this embodiment, after the execution subject acquires the change range, it can further determine whether the change range meets the preset condition. If the change range meets the preset condition, the change data of the pupil diameter of the user over time in the time period is acquired.

[0053] Here, since the user visual line focus area is usually a square, it can be represented by a center point and a side length. The execution subject can determine whether the change range of the user visual line focus area meets the preset condition by determining whether the change range of the center point and the side length meets the preset condition.

[0054] The pupil diameter of the user can be determined based on the image of the pupil of the user collected by the image collection device worn on the head of the user.

[0055] Here, the execution subject can determine the pupil diameter based on the image of the pupil of the user in the following ways: according to the image of the pupil of the user and a preset image of the pupil and a pupil diameter comparison table, or according to the image of the pupil of the user and a preset pupil diameter prediction model. The present application does not limit this.

[0056] It should be noted that the preset condition can be set according to experience and actual needs, for example, the change of the center point is less than 1 cm, the change of the side length is less than 1 cm, etc. The present application does not limit this.

[0057] In some optional manners, the pupil diameter change data of the user in the preset time period is acquired, including: collecting the image of the pupil of the user at each time in the preset time period via the image collection device worn on the eye of the user; for the image of the pupil of the user at each time, inputting the image of the pupil of the user into a preset pupil diameter prediction model to obtain the pupil diameter; and determining the pupil diameter change data of the user in the preset time period according to the pupil diameter.

[0058] In the implementation, the execution subject can collect images of the pupil of the user at each time in the preset time period via an image collection device worn on the eyes of the user, for example, a miniature camera arranged in smart glasses, etc.; for the image of the pupil of the user at each time, the image of the pupil of the user is input into a preset pupil diameter prediction model to obtain a pupil diameter, and according to the pupil diameter, pupil diameter change data of the user in the preset time period, i.e., change data of the pupil diameter of the user with time, is determined.

[0059] The preset pupil diameter prediction model is trained based on image samples of the pupil of the user with pupil diameters labeled.

[0060] Here, the pupil diameter labeled in the image sample of the pupil of the user can be obtained by extracting the pupil diameter size in the pupil image via Photoshop software.

[0061] The implementation collects images of the pupil of the user at each time in the preset time period via an image collection device worn on the eyes of the user; for the image of the pupil of the user at each time, the image of the pupil of the user is input into a preset pupil diameter prediction model to obtain a pupil diameter; and according to the pupil diameter, pupil diameter change data of the user in the preset time period is determined, which improves the accuracy of the determined pupil diameter change data.

[0062] In step 203, emotion information of the user to the focus area is determined based on the pupil diameter change data.

[0063] In the embodiment, after the execution subject determines change data of the pupil diameter with time in the preset time period according to the pupil diameter size data at each time in the preset time period, the execution subject can determine emotion information of the user to the focus area according to the pupil diameter change data and a preset pupil diameter change data-emotion information comparison table, or determine emotion information of the user to the focus area in the preset time period according to the pupil diameter change data and a preset emotion prediction model, which is not limited in the application.

[0064] Here, the emotion information can include multiple types, for example, happy, calm, anxious, and impatient, etc.

[0065] Specifically, in the preset time period, for example, 1-T, change data of the pupil diameter of the user with time is that at time 1, the pupil diameter is 2.5 mm; at time t1, the pupil diameter is 2.6 mm; at time t2, the pupil diameter is 2.7 mm; and at time T, the pupil diameter is 2.8 mm. After the execution subject obtains the change data of the pupil diameter of the user with time, the execution subject can input the change data of the pupil diameter with time into a preset emotion prediction model to obtain emotion information of the user.

[0066] In some optional manners, the emotion information of the user to the focus area is determined based on the pupil diameter change data, including: inputting the pupil diameter change data into a preset emotion prediction model to obtain the emotion information of the user to the focus area.

[0067] In the implementation manner, the execution subject can input the pupil diameter change data into the preset emotion prediction model to generate the emotion information of the user to the focus area.

[0068] The preset emotion prediction model can be trained based on pupil diameter change data samples labeled with emotion information.

[0069] Here, the preset emotion prediction model can be a deep learning model in the prior art or future development technology, for example, RNN (Recurrent Neural Network), LSTM (Long-Short Term Memory), etc., which is not limited in the present application.

[0070] Specifically, the execution subject can input the pupil diameter change data in the preset time period into the preset emotion prediction model, such as an LSTM model, to obtain the probability of four emotional states of happy, calm, anxious and impatient, and determine the emotion information of the user to the focus area based on the probabilities of the four emotional states.

[0071] Continuing to refer to Figure 5 , Figure 5 is one of the application scenarios of the information generation method according to the embodiment.

[0072] In the application scenario of Figure 5 , the execution subject 501 can determine the focus area 505 of the user's line of sight on the smart display device 504 via a positioning device worn on the head of the user 502, such as smart glasses 503 provided with a UWB positioning chip, a positioning device provided on the smart display device 504, an optical axis, an orthoview axis, etc., and obtain the change range of the focus area 505 of the user's line of sight on the smart display device 504 in a preset time period. In response to determining that the change range meets the preset condition, the execution subject can collect images of the user's pupils at each time in the preset time period via an image collection device, such as a miniature camera provided on the smart glasses, and determine the pupil diameter change data of the user in the preset time period according to the images of the user's pupils, and further determine the emotion information of the user to the focus area based on the pupil diameter change data.

[0073] The embodiment of the disclosure provides an information generation method. The information generation method comprises the following steps: acquiring a change range of a focusing area of a user's line of sight on a smart display device in a preset time period; in response to determining that the change range meets a preset condition, acquiring pupil diameter change data of the user in the preset time period; and determining emotional information of the user on the focusing area based on the pupil diameter change data. The embodiment of the disclosure overcomes the problem in the prior art that the user's emotion is easily disturbed by the environment according to the pupil size at a single point in time, and effectively improves the accuracy of the emotional information generated for the focusing area.

[0074] Further reference is made to Figure 6 which shows a flow 600 of still another embodiment of the information generation method. In this embodiment, the flow 500 of the information generation method can comprise the following steps:

[0075] In step 601, a change range of a focusing area of a user's line of sight on a smart display device in a preset time period is acquired.

[0076] In this embodiment, the implementation details and technical effects of step 601 can be referred to the description of step 201, which will not be repeated here.

[0077] In step 602, in response to determining that the change range meets a preset condition, pupil diameter change data of the user in the preset time period is acquired.

[0078] In this embodiment, the implementation details and technical effects of step 602 can be referred to the description of step 202, which will not be repeated here.

[0079] In step 603, emotional information of the user on the focusing area is determined based on the pupil diameter change data.

[0080] In this embodiment, the implementation details and technical effects of step 603 can be referred to the description of step 203, which will not be repeated here.

[0081] In step 604, a processing operation corresponding to the emotional information and a current scene category is performed.

[0082] In this embodiment, after the emotional information is acquired, the subject can determine the corresponding processing operation according to the emotional information, the current scene category, and a preset emotional information and scene category and processing operation table, and perform the corresponding processing operation, or can determine the corresponding processing operation according to the emotional information, the current scene category, and a preset operation prediction model, and perform the corresponding processing operation, which is not limited in the present application.

[0083] The preset operation prediction model is trained based on emotional information and scene category samples labeled with corresponding processing operations.

[0084] In some optional manners, the processing operation corresponding to the emotional information and the current scene category is performed, including: in response to determining that the current scene category is a video program watching scene and the emotional information is a first preset emotion, determining a person included in the focus area; and pushing the related information of the person to the user.

[0085] In the implementation manner, the subject performing the processing operation can determine the emotional information and the current scene category, and if it is determined that the current scene category is a video program watching scene and the emotional information is a first preset emotion, a person included in the focus area is determined, and the related information of the person, such as identity information and work information, is pushed to the user.

[0086] The first preset emotion can be any positive emotion, such as happy, grateful, moved, and the like.

[0087] Specifically, when the user watches a video program, if it is determined that the change range of the focus area of the user's line of sight on the smart display device in a preset time period meets a preset condition, and the emotional information determined according to the pupil change data of the user in the preset time period is a first preset emotion, such as happy, the person included in the focus area, such as person X, can be determined by the cloud server, and the classic works of the leading actor of person X are recommended to the user.

[0088] The implementation manner determines the person included in the focus area in response to determining that the current scene category is a video program watching scene and the emotional information is a first preset emotion, and pushes the related information of the person to the user, which is helpful to accurately recommend information of interest to the user based on the emotional information.

[0089] In some optional manners, the processing operation corresponding to the emotional information and the current scene category is performed, including: in response to determining that the current scene category is a psychological test scene and the emotional information is a second preset emotion, outputting prompt information indicating a psychological abnormality.

[0090] In the implementation manner, the subject performing the processing operation can determine the emotional information and the current scene category, and if it is determined that the current scene category is a psychological test scene and the emotional information is a second preset emotion, prompt information indicating a psychological abnormality is outputted.

[0091] The second preset emotion is different from the preset emotion.

[0092] Specifically, in the psychological test scene, if it is determined that the change range of the focus area of the user's line of sight on the smart display device in a preset time period meets a preset condition, and the emotional information determined according to the pupil change data of the user in the preset time period is a second preset emotion, such as anxious, the second preset emotion is different from the preset emotion, such as happy, and the prompt information indicating a psychological abnormality can be outputted.

[0093] The implementation manner is helpful for detecting the user's psychology based on the emotional information by outputting the prompt information indicating the psychological abnormality in response to determining that the current scene category is a psychological test scene and the emotional information is a second preset emotion.

[0094] In some optional manners, the processing operation corresponding to the emotional information and the current scene category is performed, including: in response to determining that the current scene category is a driving scene and the emotional information is a third preset emotion, outputting alarm information.

[0095] In the implementation manner, the subject performing the processing operation can determine the emotional information and the current scene category, and output the alarm information in response to determining that the current scene category is a driving scene and the emotional information is a third preset emotion.

[0096] The third preset emotion can be set according to experience and actual needs, for example, anxiety, excitement, etc.

[0097] Specifically, in the driving scene, if it is determined that a change range of a focusing area of a user's line of sight on the intelligent display device in a preset time period meets a preset condition, and the emotional information determined according to the pupil change data of the user in the preset time period is a third preset emotion, for example, anxiety, the alarm information can be output to prompt the user to drive carefully.

[0098] The implementation manner is helpful for improving the safety of user driving by outputting the alarm information in response to determining that the current scene category is a driving scene and the emotional information is a third preset emotion.

[0099] In some optional manners, the processing operation corresponding to the emotional information and the current scene category is performed, including: in response to determining that the current scene category is one of the following scenes: a virtual reality scene, an augmented reality scene, and a meta universe scene, and the emotional information is a fourth preset emotion, increasing a rendering quality level of a picture of the focusing area by a preset level.

[0100] In the implementation manner, the subject performing the processing operation can determine the emotional information and the current scene category, and increase the rendering quality level of the picture of the focusing area by the preset level in response to determining that the current scene category is one of the following scenes: a virtual reality scene, an augmented reality scene, and a meta universe scene, and the emotional information is a fourth preset emotion.

[0101] The fourth preset emotion can be set according to experience and actual needs, for example, joy, excitement, etc.

[0102] Specifically, in the virtual reality scene, if it is determined that the change range of the focus area of the user's line of sight on the smart display device in a preset time period meets a preset condition, and the emotion information determined according to the pupil change data of the user in the preset time period is a fourth preset emotion, for example, joy, the rendering quality level of the picture of the focus area is increased by a preset level to enhance the rendering effect.

[0103] In addition, the execution subject can also determine the character included in the picture of the focus area, and determine the fourth preset emotion as the emotion of the user to the character included in the picture.

[0104] The implementation manner increases the rendering quality level of the picture of the focus area by a preset level in response to determining that the current scene category is one of the following scenes: a virtual reality scene, an augmented reality scene, and a meta-universe scene, and the emotion information is a fourth preset emotion, thereby effectively improving the interaction effect in the virtual reality environment.

[0105] For the above implementation manner, the first preset emotion, the second preset emotion, the third preset emotion, and the fourth preset emotion can be the same or different, and no limitation is applied thereto.

[0106] The above embodiments of the present application, and Figure 2 Compared with the corresponding embodiments, the flow 600 of the information generation method in the present embodiment embodies determining the emotion information of the user to the focus area based on the pupil diameter change data, and performing a processing operation corresponding to the emotion information and the current scene category, which helps to perform a targeted processing operation according to the emotion information.

[0107] Further reference Figure 7 As an implementation of the method shown in the above figures, the present application provides an embodiment of an information generation device, which corresponds to the method embodiment shown in Figure 1 The device can be specifically applied to various electronic devices.

[0108] As shown in Figure 7 The information generation device 700 of the present embodiment includes a first acquisition module 701, a second acquisition module 702, and a determination emotion module 703.

[0109] The first acquisition module 701 can be configured to acquire the change range of the focus area of the user's line of sight on the smart display device in a preset time period.

[0110] The second acquisition module 702 can be configured to acquire the pupil diameter change data of the user in the preset time period in response to determining that the change range meets a preset condition.

[0111] The determining emotion module 703 can be configured to determine emotion information of the user on the focus area based on the pupil diameter change data.

[0112] In some optional manners of the present embodiment, the device further comprises an operation performing module which can be configured to perform a processing operation corresponding to the emotion information and the current scene category.

[0113] In some optional manners of the present embodiment, the operation performing module is further configured to: in response to determining that the current scene category is a video program watching scene and the emotion information is a first preset emotion, determine a person contained in the focus area; and push relevant information of the person to the user.

[0114] In some optional manners of the present embodiment, the operation performing module is further configured to: in response to determining that the current scene category is a psychological test scene and the emotion information is a second preset emotion, output prompt information indicating a psychological abnormality.

[0115] In some optional manners of the present embodiment, the operation performing module is further configured to: in response to determining that the current scene category is a driving scene and the emotion information is a third preset emotion, output alarm information.

[0116] In some optional manners of the present embodiment, the operation performing module is further configured to: in response to determining that the current scene category is one of the following scenes: a virtual reality scene, an augmented reality scene, and a meta universe scene, and the emotion information is a fourth preset emotion, increase a rendering quality level of a picture of the focus area by a preset level.

[0117] In some optional manners of the present embodiment, the focus area is determined by: determining an optical axis based on the first coordinate, the second coordinate, and a coordinate difference value between the preset user pupil coordinate and the first coordinate; determining a center of the focus area according to a first intersection point of the optical axis and the smart display device; determining a second intersection point of the optical axis and the preset visual axis angle, and determining a distance between the first intersection point and the second intersection point as a half side length of the focus area; and determining the focus area according to the side length and the center of the focus area.

[0118] In some optional manners of the present embodiment, the second obtaining module is further configured to: collect images of the user's pupil at each time point in a preset time period via an image collection device worn on the user's eye; for the image of the user's pupil at each time point, input the image of the user's pupil into a preset pupil diameter prediction model to obtain a pupil diameter; and determine the pupil diameter change data of the user in the preset time period according to the pupil diameter.

[0119] In some optional embodiments of this example, the emotion determination module is further configured to: input pupil diameter change data into a preset emotion prediction model to obtain the user's emotion information regarding the focused area.

[0120] According to embodiments of this application, this application also provides smart glasses and a readable storage medium.

[0121] like Figure 8 The diagram shown is a block diagram of smart glasses based on an information generation method according to an embodiment of this application.

[0122] 800 is a block diagram of smart glasses according to an embodiment of the information generation method of this application. For example... Figure 8 As shown, the smart glasses include one or more processors 801, a memory 802, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the smart glasses, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple smart glasses can be connected, each device providing some of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). Figure 8 Take the 801 processor as an example.

[0123] The memory 802 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the information generation method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the information generation method provided in this application.

[0124] Memory 802, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the information generation method in the embodiments of this application (e.g., appendix). Figure 7 The first acquisition module 701, the second acquisition module 702, and the emotion determination module 703 are shown. The processor 801 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 802, thereby implementing the information generation method in the above method embodiments.

[0125] The memory 802 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs, and / or data required by at least one function. The data storage area can store data created by the information generating smart glasses, such as information generated by the information generating smart glasses. In addition, the memory 802 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid state memory device. In some embodiments, the memory 802 can optionally include a memory disposed remotely from the processor 801, which can be connected to the information generating smart glasses through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0126] The information generating smart glasses can also include an input device 803 and an output device 804. The processor 801, the memory 802, the input device 803, and the output device 804 can be connected by a bus or other means, Figure 8

[0127] The input device 803 can receive input digital or character information, and generate key signal inputs related to the quality monitoring of the live video stream and the user settings and function controls of the information generating smart glasses, such as touch screens, keypads, mice, trackpads, touchpads, pointing sticks, one or more mouse buttons, trackballs, joysticks, and the like. The output device 804 can include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), and the like. The display devices can include, but are not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some implementations, the display devices can be touch screens.

[0128] Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0129] ​These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms "machine-readable medium" "computer-readable medium" refers to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0130] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0131] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0132] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0133] According to the technical scheme of the embodiment of the present application, the accuracy of the emotion information for the focus area is effectively improved.

[0134] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in this application can be performed in parallel, in series, in a different order, or any combination thereof, as long as the desired results of the technology disclosed in this application are achieved, which is not limited herein.

[0135] The specific embodiments described above are not to be taken as limitations on the scope of the application. It is to be understood that various modifications, combinations, sub-combinations, and alternatives can be employed, as indicated by the spirit and principles of the application. Any and all modifications, equivalents, or alternatives falling within the spirit and principles of the application are intended to be included within the scope of the application.

Claims

1. An information generation method, comprising: obtaining a change range of a focus area of a user's line of sight on a smart display device in a preset time period, the focus area being determined by: determining an optical axis based on a first coordinate, a second coordinate, and a preset coordinate difference between a user's pupil coordinate and the first coordinate, wherein the first coordinate is used to indicate a coordinate determined according to a positioning device worn on the user's eye, and the second coordinate is used to indicate a coordinate determined according to a positioning device arranged on the smart display device; determining a first intersection point of the optical axis and the smart display device as a center of the focus area; determining a second intersection point of a visual axis and the smart display device according to the optical axis and a preset visual axis angle, and determining a distance between the first intersection point and the second intersection point as a half side length of the focus area; and determining the focus area according to the side length and the center of the focus area, wherein the visual axis angle can include a direct visual axis angle and an oblique visual axis angle; in response to determining that the change range meets a preset condition, obtaining pupil diameter change data of the user in the preset time period; based on the pupil diameter change data, determining emotional information of the user on the focus area.

2. The method of claim 1, further comprising: performing a processing operation corresponding to the emotional information and a current scene category.

3. The method of claim 2, wherein, The processing operation corresponding to the emotional information and the current scene category includes: in response to determining that the current scene category is a video program watching scene and the emotional information is a first preset emotion, determining a character included in the focus area; pushing related information of the character to the user.

4. The method of claim 2, wherein, The processing operation corresponding to the emotional information and the current scene category includes: in response to determining that the current scene category is a psychological test scene and the emotional information is a second preset emotion, outputting prompt information indicating a psychological abnormality, wherein the second preset emotion is different from a preset emotion.

5. The method of claim 2, wherein, The processing operation corresponding to the emotional information and the current scene category includes: in response to determining that the current scene category is a driving scene and the emotional information is a third preset emotion, outputting warning information.

6. The method of claim 2, wherein, The processing operation corresponding to the emotional information and the current scene category includes: in response to determining that the current scene category is one of the following scenes: a virtual reality scene, an augmented reality scene, and a meta universe scene, and the emotional information is a fourth preset emotion, increasing a rendering quality level of a picture of the focus area by a preset level.

7. The method of claim 1, wherein, The obtaining of the pupil diameter change data of the user in the preset time period includes: collecting images of the user's pupil at each time in the preset time period via an image collection device worn on the user's eye; for the image of the user's pupil at each time, inputting the image of the user's pupil into a preset pupil diameter prediction model to obtain a pupil diameter, wherein the preset pupil diameter prediction model is trained based on image samples of the user's pupil labeled with pupil diameters; determining the pupil diameter change data of the user in the preset time period according to the pupil diameter.

8. The method of claim 1, wherein determining the emotional information of the user on the focus area based on the pupil diameter change data comprises: inputting the pupil diameter change data into a preset emotional prediction model to obtain the emotional information of the user on the focus area, wherein the preset emotional prediction model is trained based on pupil diameter change data samples labeled with emotional information.

9. An information generation apparatus, the apparatus comprising: a first obtaining module configured to obtain a change range of a focus area of a user's line of sight on a smart display device in a preset time period, the focus area being determined by: determining an optical axis based on a first coordinate, a second coordinate, and a preset coordinate difference value between a user's pupil coordinate and the first coordinate, wherein the first coordinate is used to indicate a coordinate determined according to a positioning device worn on the user's eye, and the second coordinate is used to indicate a coordinate determined according to a positioning device arranged on the smart display device; determining a first intersection point of the optical axis and the smart display device as a center of the focus area; determining a second intersection point of a visual axis and the smart display device according to the optical axis and a preset visual axis angle, and determining a distance between the first intersection point and the second intersection point as a half side length of the focus area, wherein the visual axis angle can include a direct visual axis angle and an oblique visual axis angle; determining the focus area according to the side length and the center of the focus area; a second obtaining module configured to obtain pupil diameter change data of the user in the preset time period in response to determining that the change range meets a preset condition; an emotional information determining module configured to determine emotional information of the user on the focus area based on the pupil diameter change data.

10. An intelligent eyewear, characterized in that, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores computer instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

11. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, the computer instructions are used to enable the computer to perform the method of any one of claims 1-8.

Images