Method and apparatus for generating information

Abstract

The present application discloses a method and device for generating information. The specific implementation scheme is: from an image to be detected including a human body image, determine the position coordinates of a head-shoulder frame corresponding to the human body image and the head-shoulder image corresponding to the position coordinates of the head-shoulder frame; based on the position coordinates of the head-shoulder image, the image to be detected and the head-shoulder frame, analyze the user corresponding to the human body image in the image to be detected, and obtain the user behavior data corresponding to the human body image in the image to be detected, wherein the user behavior data includes: the user identification of the user, the attribute information of the user, the position coordinates of the user's location, the user's gaze point information and the timestamp, and the user's gaze point information is generated by inputting the position coordinates of the head-shoulder image, the image to be detected and the head-shoulder frame into a pre-trained gaze point prediction model. The scheme locates the user through computer vision technology and obtains the user's gaze point position using image processing technology, thereby realizing the user behavior detection method.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer technology, specifically to the field of image processing technology, and in particular to a method and device for generating information. Background Art

[0002] In offline scenarios, the behavior analysis of in-store users is of great guiding significance to merchants. However, offline areas are different from online areas, and the behaviors that offline customers express their concerns are often difficult to obtain. Object detection based on deep learning is a core technology of computer vision and a core method in the application of various intelligent devices. How to use deep learning object detection technology to analyze the behavior of in-store users in offline areas has become one of our research directions.

[0003] There are currently several ways to obtain user attention in an offline area: one is to infer based on sales. During the entire shopping process, only the final settlement data of the user can be obtained. The operator infers the purchase preferences of users in the area from the final sales. Another is to use a questionnaire survey, relying on offline promoters to conduct one-on-one questionnaire surveys to obtain users' purchase preferences. Another is for the operator to judge the purchase preferences of users in the area based on personal experience. Summary of the invention

[0004] The present application provides a method, apparatus, device and storage medium for generating information.

[0005] According to a first aspect of the present application, a method for generating information is provided, the method comprising: determining, from an image to be detected including a human body image, the position coordinates of a head-and-shoulders frame corresponding to the human body image and the head-and-shoulders image corresponding to the position coordinates of the head-and-shoulders frame, wherein the image to be detected is one of several images to be detected in a certain area; based on the head-and-shoulders image, the image to be detected and the position coordinates of the head-and-shoulders frame, analyzing the user corresponding to the human body image in the image to be detected, and obtaining user behavior data corresponding to the human body image in the image to be detected, wherein the user behavior data comprises: a user identification of the user, attribute information of the user, position coordinates of the user's location, gaze point information of the user and a timestamp, the gaze point information of the user is generated by inputting the position coordinates of the head-and-shoulders image, the image to be detected and the head-and-shoulders frame into a pre-trained gaze point prediction model, the gaze point prediction model being used to characterize the detection of the position point coordinates of the user's eye gaze.

[0006] In some embodiments, determining the position coordinates of a head and shoulder border corresponding to the human body image and the head and shoulder image corresponding to the position coordinates of the head and shoulder border from an image to be detected that includes a human body image, includes: inputting the image to be detected into a pre-trained position detection model to generate the position coordinates of the head and shoulder border corresponding to the human body image in the image to be detected, wherein the position detection model is used to characterize the position detection of a combination of the human head and the human shoulder of the human body image in the image to be detected; and determining the head and shoulder image corresponding to the position coordinates of the head and shoulder border in the image to be detected based on the generated position coordinates of the head and shoulder border and the image to be detected.

[0007] In some embodiments, the gaze point prediction model is trained in the following manner: obtaining a training sample set, wherein the training samples in the training sample set include an image to be detected, the position coordinates of a head-shoulder border corresponding to a human image in the image to be detected, a head-shoulder image corresponding to the position coordinates of the head-shoulder border, and gaze point information of a user corresponding to the human image in the image to be detected; using a machine learning algorithm, taking the image to be detected, the position coordinates of a head-shoulder border corresponding to a human image in the image to be detected, and the head-shoulder image corresponding to the position coordinates of the head-shoulder border included in the training samples in the training sample set as input data, and taking the gaze point information of the user corresponding to the human image in the image to be detected as expected output data, to train the gaze point prediction model.

[0008] In some embodiments, the gaze point prediction model includes: a first feature extraction sub-model, a second feature extraction sub-model and a gaze point prediction sub-model; the process of generating the user's gaze point information includes: inputting the image to be detected and the mask image corresponding to the image to be detected into the first feature extraction sub-model, generating a first feature map corresponding to the image to be detected, wherein the first feature extraction sub-model is used to characterize the feature information extraction for the superimposed image of the image to be detected and the mask image, and the mask image is obtained by masking the image to be detected based on the position coordinates of the head and shoulder border; inputting the head and shoulder image into the second feature extraction sub-model, generating a second feature map corresponding to the head and shoulder image; superimposing the feature information in the first feature map and the feature information in the second feature map to obtain a superimposed third feature map; inputting the third feature map into the gaze point prediction sub-model, generating the user's gaze point information corresponding to the human body image in the image to be detected.

[0009] In some embodiments, before the third feature map is input into the gaze point prediction sub-model to generate the gaze point information of the user corresponding to the human body image in the image to be detected, it also includes: judging whether there is an object class in the third feature map, wherein the object class is obtained based on classifying the objects that the user's eyes are fixated on; in response to the existence of the object class in the third feature map, jumping to input the third feature map into the gaze point prediction sub-model to generate the gaze point information of the user corresponding to the human body image in the image to be detected; in response to the absence of the object class in the third feature map, outputting the judgment result and stopping the operation.

[0010] In some embodiments, the method also includes: determining an attention duration dataset and an attention degree dataset of each dimension in the region based on user behavior data corresponding to each image to be detected in the region, wherein the attention duration dataset and the attention degree dataset of each dimension at least include: an attention degree analysis of each position coordinate in the region based on all user behavior data and an attention degree analysis of users corresponding to each attribute information in the region based on all user behavior data; generating a duration graph corresponding to the attention duration dataset of each dimension and a heat graph corresponding to the attention degree dataset of each dimension based on the attention duration dataset and the attention degree dataset of each dimension.

[0011] In some embodiments, the method also includes: based on object identification, various heat maps and various duration maps, determining the attention of each object in each heat map, the duration of each object, the attention ranking corresponding to the attention of each object and the duration ranking corresponding to the duration of each object.

[0012] In some embodiments, the method also includes: determining the driving trajectory of each user in the area based on the user behavior data corresponding to each image to be detected in the area, wherein the driving trajectory is obtained by analyzing the behavior data of each user based on the user identification; and generating a driving trajectory map of each user based on the driving trajectory of each user.

[0013] In some embodiments, the method further includes: generating a user behavior report for the area and outputting the user behavior report.

[0014] According to a second aspect of the present application, a device for generating information is provided, the device comprising: a first determination unit, configured to determine, from an image to be detected including a human body image, position coordinates of a head-and-shoulders frame corresponding to the human body image and a head-and-shoulders image corresponding to the position coordinates of the head-and-shoulders frame, wherein the image to be detected is one of several images to be detected in a certain area; a first generation unit, configured to analyze a user corresponding to the human body image in the image to be detected based on the head-and-shoulders image, the image to be detected and the position coordinates of the head-and-shoulders frame, and obtain user behavior data corresponding to the human body image in the image to be detected, wherein the user behavior data comprises: a user identification of the user, attribute information of the user, position coordinates of the user's location, gaze point information of the user and a timestamp, the gaze point information of the user is generated by inputting the position coordinates of the head-and-shoulders image, the image to be detected and the head-and-shoulders frame into a pre-trained gaze point prediction model, the gaze point prediction model being used to characterize the detection of the position point coordinates of the user's eye gaze.

[0015] In some embodiments, the first determination unit includes: a position generation module, configured to input the image to be detected into a pre-trained position detection model, and generate position coordinates of a head and shoulder border corresponding to the human image in the image to be detected, wherein the position detection model is used to characterize the position detection of a combination of a human head and a human shoulder in the human image in the image to be detected; and an image determination module, configured to determine the head and shoulder image in the image to be detected corresponding to the position coordinates of the head and shoulder border based on the generated position coordinates of the head and shoulder border and the image to be detected.

[0016] In some embodiments, the gaze point prediction model in the first generation unit is trained using the following modules: a sample acquisition module, configured to acquire a training sample set, wherein the training samples in the training sample set include the image to be detected, the position coordinates of the head and shoulder border corresponding to the human image in the image to be detected, the head and shoulder image corresponding to the position coordinates of the head and shoulder border, and the gaze point information of the user corresponding to the human image in the image to be detected; a training module, configured to use a machine learning algorithm, taking the image to be detected, the position coordinates of the head and shoulder border corresponding to the human image in the image to be detected, and the head and shoulder image corresponding to the position coordinates of the head and shoulder border included in the training samples in the training sample set as input data, and taking the gaze point information of the user corresponding to the human image in the image to be detected as expected output data, to train the gaze point prediction model.

[0017] In some embodiments, the gaze point prediction model in the first generation unit includes: a first feature extraction sub-model, a second feature extraction sub-model and a gaze point prediction sub-model; the gaze point information of the user in the first generation unit is generated using the following modules: a first feature extraction module, configured to input the image to be detected and the mask image corresponding to the image to be detected into the first feature extraction sub-model, and generate a first feature map corresponding to the image to be detected, wherein the first feature extraction sub-model is used to characterize the feature information extraction for the superimposed image of the image to be detected and the mask image, and the mask image is obtained by masking the image to be detected based on the position coordinates of the head and shoulder border; the second feature extraction module, configured to input the head and shoulder image into the second feature extraction sub-model, and generate a second feature map corresponding to the head and shoulder image; the feature superposition module, configured to superimpose the feature information in the first feature map with the feature information in the second feature map, and obtain a superimposed third feature map; the gaze point prediction module, configured to input the third feature map into the gaze point prediction sub-model, and generate the user's gaze point information corresponding to the human body image in the image to be detected.

[0018] In some embodiments, the first generation unit also includes: a judgment module, configured to judge whether there is an object class in the third feature map, wherein the object class is obtained based on classifying the objects that the user's eyes are fixating on; a jump module, configured to jump to input the third feature map into the gaze point prediction sub-model in response to the existence of the object class in the third feature map, and generate the user's gaze point information corresponding to the human body image in the image to be detected; and an output module, configured to output the judgment result and stop running in response to the absence of the object class in the third feature map.

[0019] In some embodiments, the device also includes: a second determination unit, configured to determine an attention duration data set and an attention degree data set of each dimension in the area based on user behavior data corresponding to each image to be detected in the area, wherein the attention duration data set and the attention degree data set of each dimension at least include: an attention degree analysis of each position coordinate in the area based on all user behavior data and an attention degree analysis of users corresponding to each attribute information in the area based on all user behavior data; a second generation unit, configured to generate a duration graph corresponding to the attention duration data set of each dimension and a heat graph corresponding to the attention degree data set of each dimension based on the attention duration data set and the attention degree data set of each dimension.

[0020] In some embodiments, the device also includes: a third determination unit, configured to determine the attention of each object in each heat map, the duration of each object, the attention ranking corresponding to the attention of each object, and the duration ranking corresponding to the duration of each object based on the object identification, each heat map and each duration map.

[0021] In some embodiments, the device also includes: a fourth determination unit, configured to determine the driving trajectory of each user in the area based on the user behavior data corresponding to each image to be detected in the area, wherein the driving trajectory is obtained by analyzing the behavior data of each user based on the user identification; a third generation unit, configured to generate a driving trajectory map of each user based on the driving trajectory of each user.

[0022] In some embodiments, the apparatus further includes: a fourth generating unit configured to generate a user behavior report of the area and output the user behavior report.

[0023] According to the third aspect of the present application, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation manner in the first aspect.

[0024] According to a fourth aspect of the present application, the present application provides a non-transitory computer-readable storage medium storing computer instructions, characterized in that the computer instructions are used to enable a computer to execute a method as described in any implementation manner in the first aspect.

[0025] According to the technology of the present application, the position coordinates of the head and shoulders frame corresponding to the human body image and the head and shoulders image corresponding to the position coordinates of the head and shoulders frame are determined from the image to be detected including the human body image. Based on the head and shoulders image, the image to be detected and the position coordinates of the head and shoulders frame, the user corresponding to the human body image in the image to be detected is analyzed to obtain user behavior data corresponding to the human body image in the image to be detected, wherein the user behavior data includes: a user identification of the user, attribute information of the user, position coordinates of the user's location, gaze point information of the user and a timestamp. The gaze point information of the user is generated by inputting the position coordinates of the head and shoulders image, the image to be detected and the head and shoulders frame into a pre-trained gaze point prediction model. The gaze point prediction model is used to characterize the detection of the position point coordinates of the user's eye gaze. The problem that the judgment made by the reverse deduction method based on the sales situation in the prior art is biased, and it is easy to be biased towards a single type of hot-selling products and lose the opportunity to explore new growth products is solved. The problem that the questionnaire method consumes huge manpower and has a small collection range is avoided, and the problem that it is difficult to conduct manual analysis one by one for large-scale offline areas by using empirical judgment is avoided. By using images in the area to predict the coordinates of the user's eye gaze point and user behavior data, a user behavior detection method for images is implemented. By using computer vision technology and image processing technology to locate the user and capture the user's gaze point position, the most difficult to obtain user attention behavior in offline areas is captured.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0028] Figure 1 is a schematic diagram of a first embodiment of a method for generating information according to the present application;

[0029] Figure 2 It is a diagram within a region in which the method for generating information of an embodiment of the present application can be implemented;

[0030] Figure 3 is a schematic diagram of a second embodiment of a method for generating information according to the present application;

[0031] Figure 4 is a schematic diagram of a third embodiment of a method for generating information according to the present application;

[0032] Figure 5 is a structural schematic diagram of an embodiment of a device for generating information according to the present application;

[0033] Figure 6 It is a block diagram of an electronic device used to implement the method for generating information in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] Figure 1 A schematic diagram 100 of a first embodiment of a method for generating information according to the present application is shown. The method for generating information comprises the following steps:

[0037] Step 101: Determine the position coordinates of a head-shoulder frame corresponding to the human body image and the head-shoulder image corresponding to the position coordinates of the head-shoulder frame from an image to be detected including a human body image.

[0038] In this embodiment, the execution subject (such as a service platform or a terminal device) determines the position coordinates of the head and shoulder frame corresponding to the human body image of the image to be detected according to a preset frame position algorithm, and then generates a head and shoulder image corresponding to the position coordinates of the head and shoulder frame according to the position coordinates of the head and shoulder frame. The image to be detected is one of several images to be detected in a certain area. The image to be detected can be obtained by collecting the video stream of the camera. The head and shoulder frame is used to characterize the image frame containing the human head and human shoulders in the human image.

[0039] Step 102: Based on the head-shoulder image, the image to be detected and the position coordinates of the head-shoulder frame, the user corresponding to the human body image in the image to be detected is analyzed to obtain user behavior data corresponding to the human body image in the image to be detected.

[0040] In this embodiment, the execution subject can analyze the user corresponding to the human body image in the image to be detected based on the position coordinates of the head and shoulder image, the image to be detected and the head and shoulder frame obtained in step 101, using a preset user analysis method, and obtain user behavior data corresponding to the human body image in the image to be detected. The user behavior data obtained after the analysis includes: the user identification of the user, the attribute information of the user, the position coordinates of the user's location, the user's gaze point information and the timestamp. The user's gaze point information is generated by inputting the position coordinates of the head and shoulder image, the image to be detected and the head and shoulder frame into a pre-trained gaze point prediction model. The gaze point prediction model is used to characterize the detection of the position point coordinates of the user's eye gaze. The attribute information may include: attribute information such as gender and age, and the timestamp is the current time information.

[0041] In some optional implementations of the present embodiment, the gaze point prediction model is trained in the following manner: obtaining a training sample set, wherein the training samples in the training sample set include an image to be detected, the position coordinates of a head-shoulder frame corresponding to a human image in the image to be detected, a head-shoulder image corresponding to the position coordinates of the head-shoulder frame, and gaze point information of a user corresponding to the human image in the image to be detected; using a machine learning algorithm, taking the image to be detected, the position coordinates of a head-shoulder frame corresponding to a human image in the image to be detected, and the head-shoulder image corresponding to the position coordinates of the head-shoulder frame as input data, and taking the gaze point information of a user corresponding to the human image in the image to be detected as expected output data, and training to obtain a gaze point prediction model. Using a machine learning algorithm, the model is made more accurate, efficient, and has a wider application.

[0042] It should be noted that technicians can set the model structure of the above-mentioned gaze point prediction model according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0043] Continue to see Figure 2, the method 200 for generating information in this embodiment runs in the electronic device 201. In the field of monitoring and security, after the electronic device 201 for monitoring determines the position coordinates of the head-shoulder frame corresponding to the human body image and the head-shoulder image 202 corresponding to the position coordinates of the head-shoulder frame from the image to be detected including the human body image, the user corresponding to the human body image in the image to be detected is analyzed based on the head-shoulder image, the image to be detected and the position coordinates of the head-shoulder frame, and the user behavior data 203 corresponding to the human body image in the image to be detected is obtained. Among them, the user behavior data includes: the user identification of the user, the attribute information of the user, the position coordinates of the user's location, the user's gaze point information and the timestamp, and the user's gaze point information is generated by inputting the position coordinates of the head-shoulder image, the image to be detected and the head-shoulder frame into a pre-trained gaze point prediction model, and the gaze point prediction model is used to characterize the detection of the position point coordinates of the user's eyes.

[0044] The method for generating information provided by the above-mentioned embodiment of the present application adopts, from an image to be detected including a human body image, determining the position coordinates of a head-and-shoulders frame corresponding to the human body image and the head-and-shoulders image corresponding to the position coordinates of the head-and-shoulders frame, and analyzing the user corresponding to the human body image in the image to be detected based on the position coordinates of the head-and-shoulders image, the image to be detected and the head-and-shoulders frame, to obtain user behavior data corresponding to the human body image in the image to be detected, wherein the user behavior data includes: a user identifier of the user, attribute information of the user, position coordinates of the user's location, gaze point information of the user and a timestamp, and the gaze point information of the user is generated by inputting the position coordinates of the head-and-shoulders image, the image to be detected and the head-and-shoulders frame into a pre-trained gaze point prediction model, and the gaze point prediction model is used to characterize the detection of the position point coordinates of the user's eye gaze, so as to solve the problem that the judgment made by the reverse deduction method based on the sales situation in the prior art is biased, and is prone to be biased towards a single type of hot-selling products and loses the opportunity to explore new growth products, and avoids the problem that the questionnaire method consumes huge manpower and has a small collection scope, and the empirical judgment method is difficult to perform manual analysis one by one for large-scale offline areas. By using images in the area to predict the coordinates of the user's eye gaze point and user behavior data, a user behavior detection method for images is implemented. By using computer vision technology to locate the user and capture the user's gaze point position, the user's attention behavior, which is the most difficult to obtain in offline areas, is captured.

[0045] Further references Figure 3 , which shows a schematic diagram 300 of a second embodiment of a method for generating information. The process of the method comprises the following steps:

[0046] Step 301 : determining the position coordinates of a head-shoulder frame corresponding to the human body image and the head-shoulder image corresponding to the position coordinates of the head-shoulder frame from an image to be detected including a human body image.

[0047] In some optional implementations of the present embodiment, the position coordinates of the head-shoulder frame corresponding to the human body image and the head-shoulder image corresponding to the position coordinates of the head-shoulder frame are determined from an image to be detected that includes a human body image, including: inputting the image to be detected into a pre-trained position detection model to generate the position coordinates of the head-shoulder frame corresponding to the human body image in the image to be detected, wherein the position detection model is used to characterize the position detection of the combination of the human head and the human shoulder of the human body image in the image to be detected; based on the generated position coordinates of the head-shoulder frame and the image to be detected, determining the head-shoulder image corresponding to the position coordinates of the head-shoulder frame in the image to be detected. The position detection model can be based on a convolutional neural network. The position coordinates are generated using a pre-trained model to make information generation more accurate.

[0048] Step 302 , based on the head-shoulder image, the image to be detected and the position coordinates of the head-shoulder frame, the user corresponding to the human body image in the image to be detected is analyzed to obtain user behavior data corresponding to the human body image in the image to be detected.

[0049] In this embodiment, the execution subject can analyze the user corresponding to the human body image in the image to be detected based on the position coordinates of the head and shoulder image, the image to be detected and the head and shoulder frame using a preset user analysis method to obtain user behavior data corresponding to the human body image in the image to be detected. The user behavior data obtained after the analysis includes: the user identification of the user, the attribute information of the user, the position coordinates of the user's location, the user's gaze point information and the timestamp. The user's gaze point information is generated by inputting the position coordinates of the head and shoulder image, the image to be detected and the head and shoulder frame into a pre-trained gaze point prediction model. The gaze point prediction model is used to characterize the detection of the position point coordinates of the user's eyes. The gaze point prediction model is constructed based on a convolutional neural network.

[0050] In this embodiment, the gaze point prediction model may include: a first feature extraction sub-model, a second feature extraction sub-model and a gaze point prediction sub-model; the process of generating the user's gaze point information includes: inputting the image to be detected and the mask image corresponding to the image to be detected into the first feature extraction sub-model, generating a first feature map corresponding to the image to be detected, wherein the first feature extraction sub-model is used to characterize the feature information extraction for the superimposed image of the image to be detected and the mask image, and the mask image (mask map) is obtained by masking the image to be detected based on the position coordinates of the head and shoulder frame; inputting the head and shoulder image into the second feature extraction sub-model, generating a second feature map corresponding to the head and shoulder image, wherein the second feature extraction sub-model is used to characterize the feature information extraction of the head and shoulder image; superimposing the feature information in the first feature map with the feature information in the second feature map, obtaining a superimposed third feature map; inputting the third feature map into the gaze point prediction sub-model, generating the user's gaze point information corresponding to the human body image in the image to be detected. Wherein, the gaze point prediction sub-model is used to characterize the detection of the position point coordinates of the user's eye gaze.

[0051] In some optional implementations of this embodiment, before the third feature map is input into the gaze point prediction submodel to generate the gaze point information of the user corresponding to the human body image in the image to be detected, it also includes: judging whether there is an object class in the third feature map, wherein the object class is obtained based on the classification of the object that the user's eyes are looking at; in response to the existence of the object class in the third feature map, jumping to inputting the third feature map into the gaze point prediction submodel to generate the gaze point information of the user corresponding to the human body image in the image to be detected; in response to the absence of the object class in the third feature map, that is, the user's gaze point is not in the picture, outputting the judgment result and stopping the operation. The object class can be pre-set, and the judgment can output the result through two fully connected layers (FC layers). By judging in advance whether the user's gaze point exists in the picture, subsequent operations are performed for the image where the user's gaze point exists in the picture, thereby improving the system efficiency and the accuracy of system processing.

[0052] Step 303: Based on the user behavior data corresponding to each to-be-detected image in the region, determine an attention duration dataset and an attention degree dataset of each dimension in the region.

[0053] In this embodiment, the execution subject can determine the attention time data set and the attention data set of each dimension in the region by using the attention time and attention degree algorithm based on the user behavior data corresponding to each image to be detected in the region. Among them, the attention time data set of each dimension and the attention degree data set of each dimension at least include: the attention degree analysis of each position coordinate in the region based on all user behavior data and the attention degree analysis of the user corresponding to each attribute information in the region based on all user behavior data. The region here can be a shopping area in an offline scene, and each image to be detected is each image including a human body image in the region captured by the camera device. The dimensions include based on position coordinates and based on user attributes. The attention time and attention degree algorithm can be completed by counting the number of attention points and the cumulative attention time. The more attention points there are, the more people pay attention to them, and the longer the cumulative attention time is, the longer the user pays attention.

[0054] Step 304 , based on the attention duration datasets of each dimension and the attention degree datasets of each dimension, a duration graph corresponding to the attention duration datasets of each dimension and a heat graph corresponding to the attention degree datasets of each dimension are generated.

[0055] In this implementation, the execution entity can mark the attention duration and the attention degree on the monitoring image based on the attention duration dataset of each dimension and the attention degree dataset of each dimension, and generate a time graph corresponding to the attention duration dataset of each dimension and a heat graph corresponding to the attention degree dataset of each dimension.

[0056] In some optional implementations of this embodiment, the method further includes: determining the attention of each object in each heat map, the duration of each object, the attention ranking corresponding to the attention of each object, and the duration ranking corresponding to the duration of each object based on the object identifier, each heat map, and each duration map. It is possible to quickly and intuitively understand the attention of different objects in the area and prepare for the improvement of product selection in the area.

[0057] In some optional implementations of this embodiment, the method further includes: generating a user behavior report for a region and outputting the user behavior report, thereby helping to improve product selection strategies and perform product inventory management in the form of reports.

[0058] It should be noted that the first feature extraction sub-model, the second feature extraction sub-model and the gaze point prediction sub-model can be data tables or calculation formulas, etc. The technicians can set the model structures of the first feature extraction sub-model, the second feature extraction sub-model and the gaze point prediction sub-model according to actual needs. The embodiments of the present disclosure do not limit this.

[0059] In this embodiment, the specific operation of step 301 is the same as Figure 1 The operation of step 101 in the illustrated embodiment is substantially the same and will not be described in detail herein.

[0060] from Figure 3 It can be seen that Figure 1 Compared with the corresponding embodiment, the schematic diagram 300 of the method for generating information in this embodiment adopts inputting the image to be detected and the mask image corresponding to the image to be detected into the first feature extraction sub-model to generate a first feature map corresponding to the image to be detected, inputting the head and shoulder image into the second feature extraction sub-model to generate a second feature map corresponding to the head and shoulder image, superimposing the feature information in the first feature map and the feature information in the second feature map to obtain a superimposed third feature map, and inputting the third feature map into the gaze point prediction sub-model to generate user gaze point information in the user behavior data corresponding to the human body image in the image to be detected, based on the user behavior data corresponding to each image to be detected in the area, determining the attention time data set and the attention data set of each dimension in the area, based on the attention time data set and the attention data set of each dimension, generating a time map corresponding to the attention time data set of each dimension and a heat map corresponding to the attention data set of each dimension, obtaining the eye gaze points of users in the store through computer vision technology, and obtaining the purchasing preferences of users in offline retail scenarios as a reference to help merchants improve product selection strategies and manage inventory.

[0061] Further references Figure 4 , which shows a schematic diagram 400 of a third embodiment of a method for generating information. The process of the method comprises the following steps:

[0062] Step 401 : determining the position coordinates of a head-shoulder frame corresponding to the human body image and the head-shoulder image corresponding to the position coordinates of the head-shoulder frame from an image to be detected including a human body image.

[0063] Step 402 , based on the head-shoulder image, the image to be detected and the position coordinates of the head-shoulder frame, the user corresponding to the human body image in the image to be detected is analyzed to obtain user behavior data corresponding to the human body image in the image to be detected.

[0064] Step 403: Determine the driving trajectory of each user in the area based on the user behavior data corresponding to each image to be detected in the area.

[0065] In this implementation, the execution subject can determine the driving trajectory of each user in the area based on the user behavior data corresponding to each image to be detected in the area. The driving trajectory is obtained by analyzing the behavior data of each user based on the user identification.

[0066] Step 404: Generate a driving trajectory map of each user based on the driving trajectory of each user.

[0067] In this embodiment, the execution entity can mark the driving trajectory of each user on the monitoring image to generate a driving trajectory map of each user.

[0068] In some optional implementations of this embodiment, the method further includes: generating a user behavior report for a region and outputting the user behavior report. The report is used to help guide the layout and display of the store, and a personalized marketing strategy can be formulated for users with different attributes.

[0069] In this embodiment, the specific operations of steps 401 and 402 are the same as Figure 1 The operations of steps 101 and 102 in the illustrated embodiment are substantially the same and will not be described in detail herein.

[0070] from Figure 4 It can be seen that Figure 1 Compared with the corresponding embodiments, the schematic diagram 400 of the method for generating information in this embodiment uses user behavior data corresponding to each image to be detected in the area to determine the driving trajectory of each user in the area, and generates a driving trajectory map of each user based on the driving trajectory of each user. The eye gaze points of users in the store are obtained through computer vision technology, and the purchase preferences and action trajectories of users in offline retail scenarios are statistically obtained as a reference to help guide the layout and display of the store and formulate personalized marketing strategies for users with different attributes.

[0071] Further references Figure 5 , as a response to the above Figures 1 to 4 The present application provides an embodiment of a device for generating information. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0072] like Figure 5As shown, the device 500 for generating information in this embodiment includes: a first determination unit 501 and a first generation unit 502, wherein the first determination unit is configured to determine the position coordinates of the head and shoulder frame corresponding to the human body image and the head and shoulder image corresponding to the position coordinates of the head and shoulder frame from the image to be detected including the human body image, wherein the image to be detected is one of several images to be detected in a certain area; the first generation unit is configured to analyze the user corresponding to the human body image in the image to be detected based on the head and shoulder image, the image to be detected and the position coordinates of the head and shoulder frame, and obtain user behavior data corresponding to the human body image in the image to be detected, wherein the user behavior data includes: a user identifier of the user, attribute information of the user, position coordinates of the user's location, gaze point information of the user and a timestamp, and the gaze point information of the user is generated by inputting the position coordinates of the head and shoulder image, the image to be detected and the head and shoulder frame into a pre-trained gaze point prediction model, and the gaze point prediction model is used to characterize the detection of the position point coordinates of the user's eye gaze.

[0073] In this embodiment, the specific processing of the first determining unit 501 and the first generating unit 502 of the apparatus 500 for generating information and the technical effects thereof can be referred to in Figure 1 The relevant descriptions of step 101 to step 102 in the corresponding embodiment are not repeated here.

[0074] In some optional implementations of the present embodiment, the first determination unit includes: a position generation module, configured to input the image to be detected into a pre-trained position detection model, and generate position coordinates of a head and shoulder border corresponding to the human image in the image to be detected, wherein the position detection model is used to characterize the position detection of a combination of a human head and a human shoulder in the human image in the image to be detected; and an image determination module, configured to determine the head and shoulder image corresponding to the position coordinates of the head and shoulder border in the image to be detected based on the generated position coordinates of the head and shoulder border and the image to be detected.

[0075] In some optional implementations of the present embodiment, the gaze point prediction model in the first generation unit is trained using the following modules: a sample acquisition module, configured to acquire a training sample set, wherein the training samples in the training sample set include the image to be detected, the position coordinates of the head-shoulder border corresponding to the human image in the image to be detected, the head-shoulder image corresponding to the position coordinates of the head-shoulder border, and the gaze point information of the user corresponding to the human image in the image to be detected; a training module, configured to use a machine learning algorithm, taking the image to be detected, the position coordinates of the head-shoulder border corresponding to the human image in the image to be detected, and the head-shoulder image corresponding to the position coordinates of the head-shoulder border included in the training samples in the training sample set as input data, and taking the gaze point information of the user corresponding to the human image in the image to be detected as expected output data, to train the gaze point prediction model.

[0076] In some optional implementations of the present embodiment, the gaze point prediction model in the first generation unit includes: a first feature extraction sub-model, a second feature extraction sub-model and a gaze point prediction sub-model; the gaze point information of the user in the first generation unit is generated using the following modules: a first feature extraction module, configured to input the image to be detected and the mask image corresponding to the image to be detected into the first feature extraction sub-model, and generate a first feature map corresponding to the image to be detected, wherein the first feature extraction sub-model is used to characterize the feature information extraction for the superimposed image of the image to be detected and the mask image, and the mask image is obtained by performing image masking on the image to be detected based on the position coordinates of the head and shoulder border; the second feature extraction module, configured to input the head and shoulder image into the second feature extraction sub-model, and generate a second feature map corresponding to the head and shoulder image; the feature superposition module, configured to superimpose the feature information in the first feature map with the feature information in the second feature map, and obtain a superimposed third feature map; the gaze point prediction module, configured to input the third feature map into the gaze point prediction sub-model, and generate the gaze point information of the user corresponding to the human body image in the image to be detected.

[0077] In some optional implementations of this embodiment, the first generation unit also includes: a judgment module, configured to judge whether there is an object class in the third feature map, wherein the object class is obtained based on classifying the objects that the user's eyes are fixating on; a jump module, configured to jump to input the third feature map into the gaze point prediction sub-model in response to the existence of the object class in the third feature map, and generate the user's gaze point information corresponding to the human body image in the image to be detected; and an output module, configured to output the judgment result and stop running in response to the absence of the object class in the third feature map.

[0078] In some optional implementations of the present embodiment, the device also includes: a second determination unit, configured to determine an attention duration data set and an attention degree data set of each dimension in the area based on user behavior data corresponding to each image to be detected in the area, wherein the attention duration data set and the attention degree data set of each dimension at least include: data obtained by performing an attention degree analysis on each position coordinate in the area based on all user behavior data and data obtained by performing an attention degree analysis on users corresponding to each attribute information in the area based on all user behavior data; a second generation unit, configured to generate a duration graph corresponding to the attention duration data set of each dimension and a heat graph corresponding to the attention degree data set of each dimension based on the attention duration data set and the attention degree data set of each dimension.

[0079] In some optional implementations of this embodiment, the device also includes: a third determination unit, configured to determine the attention of each object in each heat map, the duration of each object, the attention ranking corresponding to the attention of each object, and the duration ranking corresponding to the duration of each object based on the object identification, each heat map and each duration map.

[0080] In some optional implementations of this embodiment, the device also includes: a fourth determination unit, configured to determine the driving trajectory of each user in the area based on the user behavior data corresponding to each image to be detected in the area, wherein the driving trajectory is obtained by analyzing the behavior data of each user based on the user identification; a third generation unit, configured to generate a driving trajectory map of each user based on the driving trajectory of each user.

[0081] In some optional implementations of this embodiment, the device further includes: a fourth generating unit configured to generate a user behavior report of the area and output the user behavior report.

[0082] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0083] like Figure 6 , is a block diagram of an electronic device for generating information according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0084] like Figure 6 As shown, the electronic device includes: one or more processors 601, a memory 602, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 601 is taken as an example.

[0085] The memory 602 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to perform the method for generating information provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the method for generating information provided in the present application.

[0086] The memory 602 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the method for generating information in the embodiment of the present application (for example, the attached Figure 5 The processor 601 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 602, that is, implements the method for generating information in the above method embodiment.

[0087] The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device for generating information, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 602 may optionally include a memory remotely arranged relative to the processor 601, and these remote memories may be connected to the electronic device for generating information via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0088] The electronic device for the method of generating information may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0089] The input device 603 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device used to generate information, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, and other input devices. The output device 604 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0090] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0092] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0093] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0094] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0095] According to the technical solution of the embodiment of the present application, the position coordinates of the head and shoulders frame corresponding to the human body image and the head and shoulders image corresponding to the position coordinates of the head and shoulders frame are determined from the image to be detected including the human body image. Based on the position coordinates of the head and shoulders image, the image to be detected and the head and shoulders frame, the user corresponding to the human body image in the image to be detected is analyzed to obtain user behavior data corresponding to the human body image in the image to be detected, wherein the user behavior data includes: a user identification of the user, attribute information of the user, position coordinates of the user's location, gaze point information of the user and a timestamp. The gaze point information of the user is generated by inputting the position coordinates of the head and shoulders image, the image to be detected and the head and shoulders frame into a pre-trained gaze point prediction model. The gaze point prediction model is used to characterize the detection of the position point coordinates of the user's eye gaze. The problem that the judgment made by the reverse deduction method based on the sales situation in the prior art is biased, and it is easy to be biased towards a single type of hot-selling products and lose the opportunity to explore new growth products is solved. The problem that the questionnaire method consumes huge manpower and has a small collection range is avoided, and the problem that it is difficult to conduct manual analysis one by one for large-scale offline areas by using empirical judgment is avoided. By using images in the area to predict the coordinates of the user's eye gaze point and user behavior data, a user behavior detection method for images is implemented. By using computer vision technology to locate the user and capture the user's gaze point position, the user's attention behavior, which is the most difficult to obtain in offline areas, is captured.

[0096] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0097] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for generating information, the method comprising: Determine, from an image to be detected including a human body image, position coordinates of a head-shoulder frame corresponding to the human body image and a head-shoulder image corresponding to the position coordinates of the head-shoulder frame, wherein the image to be detected is one of a plurality of images to be detected in a certain area; Based on the position coordinates of the head and shoulder image, the image to be detected and the head and shoulder frame, the user corresponding to the human body image in the image to be detected is analyzed to obtain user behavior data corresponding to the human body image in the image to be detected, wherein the user behavior data includes: a user identifier of the user, attribute information of the user, position coordinates of the user's location, gaze point information of the user and a timestamp, and the gaze point information of the user is generated by inputting the position coordinates of the head and shoulder image, the image to be detected and the head and shoulder frame into a pre-trained gaze point prediction model, and the gaze point prediction model is used to characterize the detection of the position point coordinates of the user's eye gaze.

2. The method according to claim 1, wherein: The step of determining, from the image to be detected including the human body image, the position coordinates of the head-shoulder frame corresponding to the human body image and the head-shoulder image corresponding to the position coordinates of the head-shoulder frame comprises: Input the image to be detected into a pre-trained position detection model to generate the position coordinates of the head and shoulder frame corresponding to the human image in the image to be detected, wherein the position detection model is used to characterize the position detection of the combination of the human head and the human shoulder of the human image in the image to be detected; Based on the generated position coordinates of the head-shoulder frame and the image to be detected, a head-shoulder image corresponding to the position coordinates of the head-shoulder frame in the image to be detected is determined.

3. The method according to claim 1, wherein: The gaze point prediction model is trained in the following way: Acquire a training sample set, wherein the training samples in the training sample set include the image to be detected, the position coordinates of the head-shoulder frame corresponding to the human image in the image to be detected, the head-shoulder image corresponding to the position coordinates of the head-shoulder frame, and the gaze point information of the user corresponding to the human image in the image to be detected; By using a machine learning algorithm, the training samples in the training sample set, including the image to be detected, the position coordinates of the head-shoulder frame corresponding to the human body image in the image to be detected, and the head-shoulder image corresponding to the position coordinates of the head-shoulder frame, are used as input data, and the gaze point information of the user corresponding to the human body image in the image to be detected is used as expected output data to train and obtain the gaze point prediction model.

4. The method according to any one of claims 1 and 3, wherein: The gaze point prediction model includes: a first feature extraction sub-model, a second feature extraction sub-model and a gaze point prediction sub-model; the process of generating the user's gaze point information includes: Inputting the image to be detected and the mask image corresponding to the image to be detected into the first feature extraction sub-model to generate a first feature map corresponding to the image to be detected, wherein the first feature extraction sub-model is used to characterize feature information extraction for a superimposed image of the image to be detected and the mask image, and the mask image is obtained by performing image masking on the image to be detected based on the position coordinates of the head-shoulder frame; Inputting the head-shoulder image into the second feature extraction sub-model to generate a second feature map corresponding to the head-shoulder image; Superimposing the feature information in the first feature map and the feature information in the second feature map to obtain a superimposed third feature map; The third feature map is input into the gaze point prediction sub-model to generate the gaze point information of the user corresponding to the human body image in the image to be detected.

5. The method according to claim 4, wherein: Before inputting the third feature map into the gaze point prediction sub-model to generate the gaze point information of the user corresponding to the human body image in the image to be detected, the method further includes: Determining whether there is an object class in the third feature map, wherein the object class is obtained based on classifying the object that the user's eyes are gazing at; In response to the existence of the object class in the third feature map, jumping to inputting the third feature map into the gaze point prediction sub-model to generate gaze point information of the user corresponding to the human body image in the image to be detected; In response to the object class not existing in the third feature map, the determination result is output and the operation is stopped.

6. The method according to claim 1, further comprising: Based on the user behavior data corresponding to each of the to-be-detected images in the area, determining an attention duration data set and an attention degree data set of each dimension in the area, wherein the attention duration data set and the attention degree data set of each dimension at least include: an attention degree analysis of each of the position coordinates in the area based on all of the user behavior data and an attention degree analysis of users corresponding to each of the attribute information in the area based on all of the user behavior data; Based on the attention duration datasets of the respective dimensions and the attention degree datasets of the respective dimensions, a duration graph corresponding to the attention duration datasets of the respective dimensions and a heat graph corresponding to the attention degree datasets of the respective dimensions are generated.

7. The method according to claim 6, further comprising: Based on the object identification, each of the heat maps and each of the duration maps, the attention degree of each of the objects in each of the heat maps, the duration of each of the objects, the attention degree ranking corresponding to the attention degree of each of the objects and the duration ranking corresponding to the duration of each of the objects are determined.

8. The method according to claim 1, further comprising: Determine, based on the user behavior data corresponding to each of the to-be-detected images in the area, a driving trajectory of each of the users in the area, wherein the driving trajectory is obtained by analyzing the behavior data of each of the users based on the user identifier; Based on the driving trajectory of each of the users, a driving trajectory map of each of the users is generated.

9. The method according to any one of claims 6 to 8, further comprising: A user behavior report for the area is generated and the user behavior report is output.

10. A device for generating information, the device comprising: A first determining unit is configured to determine, from an image to be detected including a human image, position coordinates of a head-shoulder frame corresponding to the human image and a head-shoulder image corresponding to the position coordinates of the head-shoulder frame, wherein the image to be detected is one of a plurality of images to be detected in a certain area; The first generating unit is configured to analyze the user corresponding to the human body image in the image to be detected based on the head and shoulder image, the image to be detected and the position coordinates of the head and shoulder frame, and obtain user behavior data corresponding to the human body image in the image to be detected, wherein the user behavior data includes: a user identifier of the user, attribute information of the user, position coordinates of the location of the user, gaze point information of the user and a timestamp, and the gaze point information of the user is generated by inputting the position coordinates of the head and shoulder image, the image to be detected and the head and shoulder frame into a pre-trained gaze point prediction model, and the gaze point prediction model is used to characterize the detection of the position point coordinates of the user's eye gaze.

11. The device according to claim 10, wherein: The first determining unit includes: A position generation module is configured to input the image to be detected into a pre-trained position detection model to generate position coordinates of a head and shoulder frame corresponding to a human image in the image to be detected, wherein the position detection model is used to characterize the position detection of a combination of a human head and a human shoulder in the human image in the image to be detected; The image determination module is configured to determine a head-shoulder image corresponding to the position coordinates of the head-shoulder frame in the image to be detected based on the generated position coordinates of the head-shoulder frame and the image to be detected.

12. The device according to claim 10, wherein: The gaze point prediction model in the first generation unit is trained using the following modules: A sample acquisition module is configured to acquire a training sample set, wherein the training samples in the training sample set include the image to be detected, the position coordinates of the head-shoulder frame corresponding to the human image in the image to be detected, the head-shoulder image corresponding to the position coordinates of the head-shoulder frame, and the gaze point information of the user corresponding to the human image in the image to be detected; The training module is configured to use a machine learning algorithm to take the image to be detected, the position coordinates of the head-shoulder frame corresponding to the human body image in the image to be detected, and the head-shoulder image corresponding to the position coordinates of the head-shoulder frame as input data, and take the gaze point information of the user corresponding to the human body image in the image to be detected as expected output data, to train and obtain the gaze point prediction model.

13. The device according to any one of claims 10 and 12, wherein: The gaze point prediction model in the first generation unit includes: a first feature extraction sub-model, a second feature extraction sub-model and a gaze point prediction sub-model; the gaze point information of the user in the first generation unit is generated by using the following modules: A first feature extraction module is configured to input the image to be detected and the mask image corresponding to the image to be detected into the first feature extraction sub-model, and generate a first feature map corresponding to the image to be detected, wherein the first feature extraction sub-model is used to characterize feature information extraction for a superimposed image of the image to be detected and the mask image, and the mask image is obtained by performing image masking on the image to be detected based on the position coordinates of the head-shoulder frame; A second feature extraction module is configured to input the head-shoulder image into the second feature extraction sub-model to generate a second feature map corresponding to the head-shoulder image; A feature superposition module is configured to superimpose the feature information in the first feature map and the feature information in the second feature map to obtain a superimposed third feature map; The gaze point prediction module is configured to input the third feature map into the gaze point prediction sub-model to generate the gaze point information of the user corresponding to the human body image in the image to be detected.

14. The device according to claim 13, wherein: The first generating unit further includes: a determination module configured to determine whether there is an object class in the third feature map, wherein the object class is obtained based on classifying the object that the user's eyes are gazing at; A jump module is configured to jump to input the third feature map into the gaze point prediction sub-model in response to the existence of the object class in the third feature map, so as to generate the gaze point information of the user corresponding to the human body image in the image to be detected; The output module is configured to output the judgment result and stop running in response to the object class not existing in the third feature map.

15. The apparatus according to claim 10, further comprising: A second determining unit is configured to determine, based on the user behavior data corresponding to each of the to-be-detected images in the area, an attention duration data set of each dimension and an attention degree data set of each dimension in the area, wherein the attention duration data set of each dimension and the attention degree data set of each dimension at least include: data obtained by analyzing the degree of attention of each of the position coordinates in the area based on all of the user behavior data and data obtained by analyzing the degree of attention of users corresponding to each of the attribute information in the area based on all of the user behavior data; The second generating unit is configured to generate a duration graph corresponding to the attention duration dataset of each dimension and a heat graph corresponding to the attention degree dataset of each dimension based on the attention duration dataset of each dimension and the attention degree dataset of each dimension.

16. The apparatus according to claim 15, further comprising: The third determination unit is configured to determine the attention of each object in each heat map, the duration of each object, the attention ranking corresponding to the attention of each object, and the duration ranking corresponding to the duration of each object based on the object identification, each heat map and each duration map.

17. The apparatus according to claim 10, further comprising: a fourth determining unit, configured to determine a driving trajectory of each of the users in the area based on the user behavior data corresponding to each of the images to be detected in the area, wherein the driving trajectory is obtained by analyzing each of the user behavior data based on the user identifier; The third generating unit is configured to generate a driving trajectory map of each user based on the driving trajectory of each user.

18. The apparatus according to any one of claims 15 to 17, further comprising: The fourth generating unit is configured to generate a user behavior report of the area and output the user behavior report.

19. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

Images