Screening method and related apparatus

Abstract

The application relates to the technical field of artificial intelligence, and provides a screening method and related equipment, which are applied to an electronic device, the electronic device comprises a shooting device, the method comprises the following steps: acquiring a face image by using the shooting device, inputting the face image into a preset key point detection model, and generating a first target detection image labeled with target key points; inputting the first target detection image into a preset gaze point regression model, obtaining a gaze point mapped on the shooting device, and determining a gaze point position of the gaze point on the shooting device; calculating a probability of the gaze point position being mapped on a target region of the shooting device, and determining a screening result according to the probability. The application can reduce screening cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a screening method and related equipment. BACKGROUND

[0002] The existing behavior observation method usually adopts an eye tracking device to collect eye movement data, and combines a screening server to obtain a screening result of a to-be-screened person. However, the detection time of this method is relatively long, and an additional eye movement collection instrument and a screening server are needed, so the screening cost is relatively high. SUMMARY

[0003] In view of the above, it is necessary to provide a screening method and related equipment to solve the technical problem of high screening cost.

[0004] A first aspect of the present application provides a screening method applied to an electronic device, wherein the electronic device comprises a photographing device, and the method comprises:

[0005] obtaining a face image by using the photographing device;

[0006] inputting the face image into a preset key point detection model to generate a first target detection image labeled with a target key point;

[0007] inputting the first target detection image into a preset gaze point regression model to obtain a gaze point mapped on the photographing device, and determining a gaze point position of the gaze point on the photographing device;

[0008] calculating a probability that the gaze point position is mapped on a target region of the photographing device, and determining a screening result according to the probability.

[0009] In some optional embodiments, before the face image is obtained by using the photographing device, the method further comprises:

[0010] obtaining training data, verification data, and a verification label corresponding to the verification data;

[0011] inputting the training data into a neural network for training according to a plurality of different preset hyperparameters to obtain an initial key point detection model corresponding to each hyperparameter;

[0012] inputting the verification data into the initial key point detection model corresponding to each hyperparameter respectively to obtain an initial key point detection result output by each initial key point detection model;

[0013] calculating a loss function value of each initial key point detection model according to each initial key point detection result and the verification label;

[0014] Determine the model with the minimum loss function value in the plurality of initial key point detection models as the trained key point detection model.

[0015] In some optional embodiments, after determining the trained key point detection model, the method further comprises:

[0016] Obtaining test data and a test label corresponding to the test data;

[0017] Inputting the test data into the trained key point detection model for testing to generate a test key point detection result;

[0018] According to the test key point detection result and the test label, calculating an error value and a fitting degree of the trained key point detection model;

[0019] If the error value is greater than a first threshold or the fitting degree is less than a second threshold, returning to perform the steps of obtaining training data and validation data;

[0020] If the error value is not greater than the first threshold and the fitting degree is not less than the second threshold, taking the trained key point detection model as the preset key point detection model.

[0021] In some optional embodiments, a plurality of test points are provided on the photographing device, and the method further comprises a step of constructing a gaze point regression model, specifically comprising:

[0022] Obtaining a viewing distance between a photographed object and the photographing device;

[0023] Image cropping is performed on images of the photographed object viewing different test points to obtain head images containing head regions of the photographed object;

[0024] Inputting the head images into the preset key point detection model to generate a second target detection image labeled with pupil key points and eye corner key points;

[0025] Calculating relative distances of the pupil key points and the eye corner key points in the second target detection image;

[0026] Based on the viewing distance, the relative distances, and positions of the test points on the photographing device, constructing the gaze point regression model.

[0027] In some optional embodiments, the pupil key points include a first pupil key point corresponding to a left eye and a second pupil key point corresponding to a right eye, the eye corner key points include a left inner eye corner key point and a right inner eye corner key point, and the calculating the relative distances of the pupil key points and the eye corner key points includes:

[0028] taking a distance between the first pupil key point and the left inner eye corner key point as a relative distance corresponding to the left eye;

[0029] taking a distance between the second pupil key point and the right inner eye corner key point as a relative distance corresponding to the right eye.

[0030] In some optional embodiments, a target region with random movement and a non-target region with random movement are arranged on the photographing device, and the calculating the probability that the gaze point position is mapped in the target region of the photographing device includes:

[0031] acquiring the gaze point position in real time based on the target region with random movement and the non-target region with random movement;

[0032] calculating a first number of times that the gaze point position is mapped in the target region;

[0033] calculating a second number of times that the gaze point position is mapped in the non-target region;

[0034] calculating a total number of times that the gaze point position is mapped on the photographing device according to the first number of times and the second number of times;

[0035] calculating the probability that the gaze point is mapped in the target region of the photographing device based on the first number of times and the total number of times.

[0036] In some optional embodiments, the determining the screening result according to the probability includes:

[0037] determining a risk score of autism of a person to be screened according to the probability that the gaze point is mapped in the target region of the photographing device, and taking the risk score as the screening result.

[0038] A second aspect of the present application provides a screening device, which includes an acquisition module, a detection module, a mapping module, and a screening module.

[0039] The acquisition module is configured to acquire a face image by using the photographing device.

[0040] The detection module is configured to input the face image into a preset key point detection model to generate a first target detection image with target key points labeled.

[0041] The mapping module is configured to input the first target detection image into a preset gaze point regression model to obtain a gaze point mapped on the shooting device and determine a gaze point position of the gaze point on the shooting device.

[0042] The screening module is configured to calculate a probability of the gaze point position being mapped on a target region of the shooting device and determine a screening result according to the probability.

[0043] A third aspect of the present application provides an electronic device, which comprises a processor and a memory, and the processor is configured to implement the screening method when executing a computer program stored in the memory.

[0044] A fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is configured to implement the screening method when executed by a processor.

[0045] The screening method and related devices provided in the present application can improve the detection accuracy of a face image through a key point detection model, further obtain a gaze point and a gaze point position mapped on a shooting device through a gaze point regression model, calculate a probability of the gaze point position being mapped on a target region of the shooting device, and obtain a screening result, thereby reducing the cost of screening. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 FIG. 1 is an application scenario diagram of the screening method provided in an embodiment of the present application.

[0047] Figure 2 FIG. 2 is a flowchart of the screening method provided in an embodiment of the present application.

[0048] Figure 3 FIG. 3 is a construction flowchart of the gaze point regression model provided in an embodiment of the present application.

[0049] Figure 4 FIG. 4 is a schematic diagram of multiple test points on a shooting device provided in an embodiment of the present application.

[0050] Figure 5 FIG. 5 is a schematic diagram of a relative distance provided in an embodiment of the present application.

[0051] Figure 6 FIG. 6 is a mapping schematic diagram of the gaze point regression model provided in an embodiment of the present application.

[0052] Figure 7 FIG. 7 is a screening process schematic diagram provided in an embodiment of the present application.

[0053] Figure 8 FIG. 8 is a structure diagram of the screening device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be described in detail below in conjunction with the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of the present application is only for the purpose of describing the embodiments of one optional implementation and is not intended to limit the present application.

[0056] The screening method provided by the embodiments of the present application can be applied to electronic devices with battery packs. The above-mentioned electronic devices can include any one of the following devices: mobile phones, tablet computers, notebook computers, smart voice interaction devices, smart home devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), household energy storage devices, mobile energy storage devices, self-moving devices, etc.

[0057] In order to better understand the screening method and related devices provided by the embodiments of the present application, the application scenarios of the battery pack capacity prediction method of the present application will be described first.

[0058] Figure 1 FIG. 1 is a schematic diagram of an application scenario of the screening method provided by the embodiments of the present application. The screening method provided by the embodiments of the present application is applied to an electronic device 1, which includes, but is not limited to, a memory 12, at least one processor 13 and a photographing device 14 which are connected to each other in communication through a communication bus 11. The photographing device 14 can be a device with photographing function, for example, a camera. The photographing device 14 is used to photograph and acquire images of a photographing object, and send the photographed images to the processor 13, so that the processor 13 performs corresponding image processing.

[0059] Figure 1 The electronic device 1 is only an example and does not constitute a limitation on the electronic device 1. In actual applications, the electronic device 1 can include more or fewer components than those shown in the figure, or combine certain components, or replace different components, for example, the electronic device 1 can also include input / output devices, network access devices, etc.

[0060] Embodiment one

[0061] The embodiment of the present application provides a screening method. Figure 2 FIG. 1 is a flowchart of a screening method provided by the embodiment of the present application. The screening method can reduce the cost of screening. According to different requirements, the order of steps in the flowchart can be changed, and some steps can be omitted.

[0062] S21, obtaining a face image by using a shooting device.

[0063] In the embodiment of the present application, the shooting device is used to shoot a shooting object, and the shooting object can be a person to be screened. The person to be screened can be shot at a specified shooting distance, or can be shot at a random shooting distance. When the person to be screened is at a random shooting distance, the actual distance between the person to be screened and the shooting device can be calculated according to the shooting parameters of the shooting device and the reference object preset in the shooting range. The shooting distance can also be referred to as a viewing distance, which refers to a specified distance or a random distance between the person to be screened and the shooting device.

[0064] The shooting device is used to shoot the person to be screened. Specifically, the shooting device can be used to directly shoot a photo of the person to be screened as an image to be detected of the person to be screened, or the shooting device can be used to record a video of the person to be screened, and then a video frame is extracted from the video recording as an image to be detected of the person to be screened.

[0065] After obtaining the image to be detected, the image to be detected can be detected by using a preset face recognition model to obtain a face image, the image to be detected can be segmented by using an image segmentation algorithm to obtain a face image, and the image to be detected can be cropped according to a preset cropping ratio to obtain a face image.

[0066] S22, inputting the face image into a preset key point detection model to generate a first target detection image labeled with target key points.

[0067] After obtaining the face image, the face image can be input into a preset key point detection model to obtain a first target detection image labeled with target key points.

[0068] The specific type of the preset key point detection model can be set according to actual needs. For example, the key point detection model can include any one or a combination of multiple types of models such as a long short-term memory (LSTM), a recurrent neural network (RNN), and a convolutional neural network (CNN). The one or more neural network combinations are trained to obtain the preset key point detection model.

[0069] In the embodiments of the present application, for the training of the preset key point detection model, a data set can be obtained by a shooting device, and the data set can be composed of multiple pictures. The data set is image pre-processed, the images in the data set are cropped into images with consistent proportions, and only the head region is retained. The cropped images are uniformly resolved by a double-line interpolation method, and the resolution can be 256x256. The pre-processed data set is divided, and is divided into training data, verification data, and a test set according to a preset ratio. The preset ratio can be 6:2:2.

[0070] For the training of the preset key point detection model, different hyperparameters (Hyperparameter) can be pre-set for the one or more combined networks mentioned above according to actual training needs, such as 0.01, 0.1, 1, 10, or 100, etc. The hyperparameter is a parameter pre-set before training of the one or more combined networks. Each pre-set hyperparameter will obtain a different preset key point detection model.

[0071] In the embodiments of the present application, based on each hyperparameter and a preset number of training times, the preset number of training times can be 1000 times. The training data is input into one or more combinations of LSTM, RNN, and CNN for training. An optimization algorithm is used to optimize the weights and biases of neurons in the back propagation process of the neural network, wherein the optimization algorithm can be an Adam algorithm, which is a gradient optimization algorithm.

[0072] When the number of training times reaches the preset number of training times, it indicates that the training is complete, and an initial key point detection model corresponding to each hyperparameter is obtained. For example, setting the hyperparameter to 0.01 and setting the hyperparameter to 0.1 can train different initial key point detection models during the training process.

[0073] After obtaining the plurality of initial key point detection models, in order to make the preset key point detection model accurately generate the target detection image labeled with the target key points, an initial key point detection model with the minimum loss function value or an initial key point detection model with the highest accuracy (Accuracy, Acc) is obtained as the trained key point detection model, and the trained key point detection model is tested by using the test data, and the model meeting the expected standard is taken as the preset key point detection model.

[0074] The initial key point detection model with the minimum loss function value specifically includes the following steps: obtaining verification data and verification labels corresponding to the verification data, the verification labels can be real values, inputting the verification data into each initial key point detection model respectively, obtaining initial key point detection results output by each initial key point detection model, calculating loss function values corresponding to each initial key point detection model according to the initial key point detection results output by each initial key point detection model and the verification labels, comparing the loss function values corresponding to each initial key point detection model, and determining the model with the minimum loss function value among the plurality of initial key point detection models as the trained key point detection model.

[0075] Alternatively, the initial key point detection model with the highest training accuracy specifically includes the following steps: obtaining verification data and verification labels corresponding to the verification data, the verification labels can be real values, inputting the verification data into each initial key point detection model respectively, obtaining initial key point detection results output by each initial key point detection model, calculating the accuracy of each initial key point detection model according to the initial key point detection results and the verification labels respectively, comparing the accuracies of the plurality of initial key point detection results, and determining the initial key point detection model with the highest accuracy as the trained key point detection model. For example, the accuracy of the initial key point detection model corresponding to the hyperparameter setting of 0.01 is 0.66, the accuracy of the initial key point detection model corresponding to the hyperparameter setting of 1 is 0.88, and the initial key point detection model corresponding to the accuracy of 0.88 is taken as the trained key point detection model.

[0076] After obtaining the trained key point detection model, in order to test whether the trained key point detection model meets the expected standard, test data and test labels corresponding to the test data are obtained, the test labels corresponding to the test data are real values of the test data, that is, real key point detection results corresponding to the test data. The test data is input into the trained key point detection model for testing, to generate test key point detection results, and to calculate error values and fitting degrees of the test key point detection results and the test labels, wherein the error values can be root mean square errors or mean absolute errors.

[0077] If the error value is greater than the first threshold value or the fitting degree is less than the second threshold value, it indicates that the trained key point detection model does not meet the expected standard, and the step of obtaining the data set is executed again. If the error value is not greater than the first threshold value and the fitting degree is not less than the second threshold value, it indicates that the trained key point detection model meets the expected standard, and the trained key point detection model meeting the expected standard is used as the preset key point detection model. The first threshold value can be 4%, and the second threshold value can be 98%.

[0078] After obtaining the preset key point detection model, the face image can be input into the preset key point detection model to obtain a target detection image labeled with target key points, including pupil key points, eye corner key points, and face contour key points.

[0079] S23, input the target detection image into the preset gaze point regression model to obtain the gaze point mapped on the shooting device and determine the gaze point position of the gaze point on the shooting device.

[0080] In the embodiments of the present application, the gaze point regression model is constructed in advance before the target detection image is input into the preset gaze point regression model.

[0081] Figure 3 is a construction flowchart of the gaze point regression model provided by the embodiments of the present application, as Figure 3 shown, the specific construction steps are as follows:

[0082] S31, obtain the viewing distance between the shooting object and the shooting device.

[0083] In the embodiments of the present application, the viewing distances of multiple shooting objects can be obtained, and each shooting object is provided with multiple viewing distances. The shooting object can view the shooting device at a preset position, and the distance between the shooting object and the shooting device is determined as the viewing distance. The viewing distance can be 30 cm, 40 cm, 50 cm, 60 cm, etc.

[0084] S32, image cropping is performed on the images of the shooting object when viewing different test points to obtain a head image containing a head region of the shooting object.

[0085] Figure 4 is a schematic diagram of multiple test points on the shooting device provided by the embodiments of the present application.

[0086] In the embodiments of the present application, as Figure 4As shown, a plurality of test points are arranged on the photographing device 14, which can include a left test point 401 on the left side of the photographing device, a right test point 402 on the right side of the photographing device, an upper test point 403 on the upper side of the photographing device, a lower test point 404 on the lower side of the photographing device, and a middle test point 405 at the center of the photographing device. The subject views the left test point, the right test point, the upper test point, the lower test point, and the middle test point on the photographing device at different viewing distances, respectively, to obtain images when viewing each test point.

[0087] After the images when viewing each test point are obtained, the images when viewing each test point are cropped to obtain head images containing head regions of the subject.

[0088] S33, input the head image into a preset key point detection model to generate a second target detection image labeled with pupil key points and eye corner key points.

[0089] In the embodiments of the present application, after the head image is obtained, the head image is input into the key point detection model trained in S22, and the key points in the head image are detected and labeled by using the key point detection model to outline the pupil, the eye socket, the eye corner, and the face contour in the head image, thereby generating a target detection image labeled with the face contour key points, the pupil key points, and the eye corner key points. When the key points of the face image are identified, the face region can be divided according to the face contour key points, thereby obtaining the pupil key points and the eye corner key points in the face region.

[0090] S34, calculate the relative distances of the pupil key points and the eye corner key points in the second target detection image.

[0091] In the embodiments of the present application, the pupil key points include a first pupil key point corresponding to the left eye and a second pupil key point corresponding to the right eye, and the eye corner key points include a left inner eye corner key point and a right inner eye corner key point. The relative distances of the pupil key points and the eye corner key points in the target detection image are calculated, including: calculating the relative distance of the left eye and the relative distance of the right eye.

[0092] Figure 5 FIG. 1 is a schematic diagram of the relative distances provided by the embodiments of the present application. As shown, the distance between the first pupil key point and the left inner eye corner key point is taken as the relative distance L1 of the left eye, and the distance between the second pupil key point and the right inner eye corner key point is taken as the relative distance R1 of the right eye. Figure 5

[0093] S35, construct a gaze point regression model based on the viewing distances, the relative distances, and the positions of the test points on the photographing device.

[0094] ​In the embodiment of the present application, the photographed object views different test points on the photographing device at each viewing distance, images of viewing different test points are obtained respectively, the relative distances of the pupil key points and the eye corner key points are calculated according to the obtained images of viewing different test points, the relationship among the viewing distance, the relative distance and the position of the test point on the photographing device is obtained, and the gaze point regression model is constructed based on the relationship among the three.

[0095] Figure 6 is a mapping schematic diagram of the gaze point regression model provided by the embodiment of the present application. As shown in Figure 6 the photographed object views different test points at a position 30 cm away from the photographing device, that is, the viewing distance d is 30 cm, the photographed object views the left test point located on the left side of the photographing device, an image of viewing the left test point is obtained, the image of viewing the left test point is input into the key point detection model, the target detection image labeled with the pupil key points and the eye corner key points is generated, and the relative distance L1 of the left eye and the relative distance R1 of the right eye are calculated according to the pupil key points and the eye corner key points. Similarly, the photographed object views the right test point located on the right side of the photographing device, an image of viewing the right test point is obtained, the image of viewing the right test point is input into the key point detection model, the target detection image labeled with the pupil key points and the eye corner key points is generated, and the relative distance L2 of the left eye and the relative distance R2 of the right eye are calculated according to the pupil key points and the eye corner key points.

[0096] The gaze point regression model is constructed according to the relationship among the calculated relative distance L1 of the left eye, the relative distance R1 of the right eye, the viewing distance d and the left test point located on the left side of the photographing device, and the relationship among the calculated relative distance L2 of the left eye, the relative distance R2 of the right eye, the viewing distance d and the right test point located on the right side of the photographing device.

[0097] The above example is only for demonstration. In the process of constructing the gaze point regression model, each photographed object views at least five test points on the photographing device at each viewing distance, and the images corresponding to each test point are obtained.

[0098] After the gaze point regression model is constructed, the first target detection image is input into the gaze point regression model to obtain the gaze point mapped on the photographing device, and the gaze point position of the gaze point on the photographing device is determined.

[0099] In the embodiment of the present application, the gaze point regression model is constructed to reduce the cost of autism screening while ensuring the screening accuracy.

[0100] S24, the probability that the gaze point position is mapped on the target region of the photographing device is calculated, and the screening result is determined according to the probability.

[0101] Figure 7is a schematic diagram of a screening process provided by an embodiment of the present application.

[0102] In the present embodiment, as shown in Figure 7 , a target area and a non-target area are provided on the camera, the target area can include a pre-set human image, the non-target area can include a pre-set object image, and the human image and the object image can move randomly on the camera. Within a pre-set time, the probability of the human image randomly moving being watched by the to-be-screened person when watching the camera is calculated, i.e., the first number of times that the gaze point position is mapped to the target area is calculated, the second number of times that the gaze point position is mapped to the non-target area is calculated, the total number of times is obtained according to the first number of times and the second number of times, and the probability of the gaze point being mapped to the human image is obtained according to the first number of times and the total number of times.

[0103] In a specific embodiment, the watching image of the to-be-screened person obtained by the camera within one minute is detected, and the human image and the object image are moved randomly at the same time, the gaze point position of the to-be-screened person mapped to the camera is detected, i.e., as shown in Figure 7 , the probability of the gaze point falling on the human image in the watching image is calculated, i.e., the ratio of the first number of times and the total number of times is calculated. Alternatively, the gaze point position falling on the human image is given 1 point and the gaze point position falling on the object image is given 0 point during the watching process.

[0104] According to the obtained probability or the calculated score, the risk of the to-be-screened person suffering from autism is obtained as the screening result of the to-be-screened person. Since the autism patient can have attention disorder, in the present embodiment, the to-be-screened person can be asked to watch the human image all the time before the autism screening, and if the to-be-screened person does not watch the human image according to the pre-set instruction, it indicates that the to-be-screened person has certain attention disorder, and then the risk of the to-be-screened person suffering from autism is obtained. In other embodiments, the to-be-screened person can also be asked to watch the object image and the like all the time.

[0105] In the embodiments of the present application, it is assumed that the obtained probability is 0.8, indicating that the risk of the to-be-screened person suffering from autism is 0.2, it is assumed that a total of 50 times are counted, and the calculated score is 50 points, indicating that the to-be-screened person is a normal person and the risk of suffering from autism is low, which can be 0.

[0106] The present application can automatically identify the preference of the to-be-screened person when watching the autism screening video, i.e., the gaze point position of the gaze point on the camera when the to-be-screened person watches, without the need for additional detection devices, thereby reducing the time cost and economic cost of screening. Based on the key point detection model obtained by training, the present application can accurately identify the target key point, thereby improving the screening accuracy.

[0107] Embodiment two

[0108] Figure 8 This is a structural diagram of the screening device provided in Embodiment 2 of this application.

[0109] In some embodiments, the screening device 800 may include a plurality of functional modules composed of computer program segments. The computer programs for each program segment in the screening device 800 may be stored in the memory of an electronic device and executed by at least one processor to perform (see details). Figure 2 (Description) Screening function.

[0110] In this embodiment, the screening device 800 can be divided into multiple functional modules according to its functions. These functional modules may include: an acquisition module 801, a detection module 802, a mapping module 803, and a screening module 804. As used in this application, a module refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory. In this embodiment, the limitations of the screening device 800 can be found in the above-described limitations of the screening method, and will not be repeated in detail here.

[0111] The acquisition module 801 is used to acquire a face image using the shooting device.

[0112] The detection module 802 is used to input the face image into a preset key point detection model to generate a first target detection image labeled with target key points.

[0113] The mapping module 803 is used to input the first target detection image into a preset gaze point regression model to obtain the gaze point mapped on the shooting device, and to determine the gaze point position of the gaze point on the shooting device.

[0114] The screening module 804 is used to calculate the probability that the gaze point position is mapped to the target area of ​​the shooting device, and determine the screening result based on the probability.

[0115] In some alternative implementations, prior to acquiring a facial image using the imaging device, the method further includes:

[0116] Obtain training data, validation data, and the corresponding validation labels for the validation data;

[0117] The training data is input into the neural network for training based on multiple pre-set hyperparameters to obtain the initial keypoint detection model corresponding to different hyperparameters;

[0118] The verification data is input into the initial keypoint detection model corresponding to different hyperparameters to obtain the initial keypoint detection result output by each initial keypoint detection model;

[0119] According to each of the initial key point detection result and the verification label, a loss function value of each of the initial key point detection model is calculated;

[0120] The loss function values of each of the initial key point detection model are compared to determine the model with the minimum loss function value in the plurality of initial key point detection models as a trained key point detection model.

[0121] In some optional embodiments, after determining the trained key point detection model, the method further comprises:

[0122] Obtaining test data and a test label corresponding to the test data;

[0123] Inputting the test data into the trained key point detection model for testing to generate a test key point detection result;

[0124] According to the test key point detection result and the test label, an error value and a fitting degree of the trained key point detection model are calculated;

[0125] If the error value is greater than a first threshold or the fitting degree is less than a second threshold, the steps of obtaining the training data and the verification data are returned to be executed;

[0126] If the error value is not greater than the first threshold and the fitting degree is not less than the second threshold, the trained key point detection model is taken as the preset key point detection model.

[0127] In some optional embodiments, a plurality of test points are arranged on the photographing device, and the method further comprises a step of constructing a gaze point regression model, specifically comprising:

[0128] Obtaining a viewing distance between a photographed object and the photographing device;

[0129] Image cropping is performed on images of the photographed object when viewing different test points to obtain a head image containing a head region of the photographed object;

[0130] The head image is input into the preset key point detection model to generate a second target detection image labeled with a pupil key point and a canthus key point;

[0131] The relative distance of the pupil key point and the canthus key point in the second target detection image is calculated;

[0132] Based on the viewing distance, the relative distance, and the positions of the test points on the photographing device, the gaze point regression model is constructed.

[0133] In some optional embodiments, the pupil key points include a first pupil key point corresponding to a left eye and a second pupil key point corresponding to a right eye, the eye corner key points include a left inner eye corner key point and a right inner eye corner key point, and the calculating the relative distances of the pupil key points and the eye corner key points includes:

[0134] taking a distance between the first pupil key point and the left inner eye corner key point as a relative distance corresponding to the left eye;

[0135] taking a distance between the second pupil key point and the right inner eye corner key point as a relative distance corresponding to the right eye.

[0136] In some optional embodiments, a target region with random movement and a non-target region with random movement are arranged on the photographing device, and the calculating the probability that the gaze point position is mapped in the target region of the photographing device includes:

[0137] acquiring the gaze point position in real time based on the target region with random movement and the non-target region with random movement;

[0138] calculating a first number of times that the gaze point position is mapped in the target region;

[0139] calculating a second number of times that the gaze point position is mapped in the non-target region;

[0140] calculating a total number of times that the gaze point position is mapped on the photographing device according to the first number of times and the second number of times;

[0141] calculating the probability that the gaze point is mapped in the target region of the photographing device based on the first number of times and the total number of times.

[0142] In some optional embodiments, the determining the screening result according to the probability includes:

[0143] determining a risk score of autism of a person to be screened according to the probability that the gaze point is mapped in the target region of the photographing device, and taking the risk score as the screening result.

[0144] Embodiment Three

[0145] This embodiment provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the screening method embodiments described above are implemented, for example Figure 2 S21-S24 shown in the figure:

[0146] S21, acquiring a face image by using the photographing device;

[0147] S22, input the face image into a preset key point detection model to generate a first target detection image with target key points labeled;

[0148] S23, input the first target detection image into a preset gaze point regression model to obtain a gaze point mapped on the shooting device and determine a gaze point position of the gaze point on the shooting device;

[0149] S24, calculate a probability of the gaze point position being mapped on a target region of the shooting device and determine a screening result according to the probability.

[0150] Alternatively, the computer program is executed by the processor to realize the functions of each module / unit in the above-mentioned device embodiments, for example Figure 8 modules 801-804 in the above-mentioned device embodiments:

[0151] The acquisition module 801 is configured to acquire a face image by using the shooting device.

[0152] The detection module 802 is configured to input the face image into a preset key point detection model to generate a target detection image with first target key points labeled.

[0153] The mapping module 803 is configured to input the first target detection image into a preset gaze point regression model to obtain a gaze point mapped on the shooting device and determine a gaze point position of the gaze point on the shooting device.

[0154] The screening module 804 is configured to calculate a probability of the gaze point position being mapped on a target region of the shooting device and determine a screening result according to the probability.

[0155] Embodiment Four

[0156] Continuing to refer to Figure 1 In the preferred embodiments of the present application, the electronic device 1 includes a memory 12, at least one processor 13, and at least one communication bus 11.

[0157] Those skilled in the art should understand that Figure 1 The structure of the electronic device shown does not constitute a limitation of the embodiments of the present application, and can be a bus structure or a star structure. The electronic device 1 can also include more or fewer other hardware or software, or different component arrangements than those shown.

[0158] In some embodiments, the electronic device 1 is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, and the hardware thereof includes but is not limited to microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors and embedded devices, etc. The electronic device 1 can also include a client device, which includes but is not limited to any electronic product capable of human-computer interaction with a client through a keyboard, a mouse, a remote controller, a touchpad or a voice control device, such as a personal computer, a tablet computer, a smart phone, a digital camera, etc.

[0159] It should be noted that the electronic device 1 is only an example, and other existing or future electronic products, such as those adaptable to the present application, should also be included in the protection scope of the present application and are hereby incorporated by reference.

[0160] In some embodiments, the memory 12 stores a computer program, and the computer program is executed by the at least one processor 13 to realize all or part of the steps of the screening method as described. The memory 12 includes a Read-Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), a One-time Programmable Read-Only Memory (OTPROM), an Electrically-Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM) or other optical disk memories, magnetic disk memories, magnetic tape memories, or any other computer-readable medium capable of carrying or storing data.

[0161] In some embodiments, the at least one processor 13 is a control unit of the electronic device 1, which connects various components of the entire electronic device 1 through various interfaces and lines, and performs various functions of the electronic device 1 and processes data by running or executing programs or modules stored in the memory 12 and calling data stored in the memory 12. For example, the at least one processor 13 implements all or part of the steps of the screening method described in the embodiments of the present application when executing the computer program stored in the memory, or implements all or part of the functions of the screening device. The at least one processor 13 can be composed of integrated circuits, for example, can be composed of a single packaged integrated circuit, or can be composed of a plurality of packaged integrated circuits with the same function or different functions, including one or more combinations of central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0162] In some embodiments, the at least one communication bus 11 is configured to realize the connection and communication between the memory 12, the at least one processor 13, and the like.

[0163] Although not shown, the electronic device 1 can further include a power supply (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, so as to realize the functions of managing charging, discharging, and power consumption management, and the like through the power management device. The power supply can also include one or more direct current or alternating current power supplies, recharging devices, power supply fault detection circuits, power supply converters or inverters, power supply status indicators, and the like. The electronic device 1 can further include various sensors, Bluetooth modules, Wi-Fi modules, camera devices, and the like, which are not described here.

[0164] The integrated units in the form of software function modules described above can be stored in a computer readable storage medium. The software function modules described above are stored in a storage medium, and include a plurality of instructions for causing a computer device (which can be a personal computer, an electronic device, or a network device, etc.) or a processor to execute part of the method described in each embodiment of the present application.

[0165] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are only schematic. For example, the division of the modules is only a logical function division. In actual implementation, there can be another division manner.

[0166] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, and may be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0167] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0168] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claims. In addition, it is obvious that the word "comprising" does not exclude other units or, singular does not exclude plural. The plurality of units or devices stated in the specification can also be realized by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, not any specific order.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A screening method applied to an electronic device, the electronic device comprising a photographing apparatus, characterized in that, The method comprises: acquiring a face image by using the photographing device, wherein a plurality of test points are arranged on the photographing device, and a target region and a non-target region that move randomly are arranged on the photographing device; inputting the face image into a preset key point detection model to generate a first target detection image labeled with target key points; obtaining a gaze point mapped on the photographing device by inputting the first target detection image into a preset gaze point regression model, and determining a gaze point position of the gaze point on the photographing device; the gaze point regression model is constructed by: acquiring a viewing distance between a photographing object and the photographing device; performing image cropping on images of the photographing object viewing different test points to obtain a head image containing a head region of the photographing object; inputting the head image into the preset key point detection model to generate a second target detection image labeled with pupil key points and eye corner key points; calculating relative distances of the pupil key points and the eye corner key points in the second target detection image; constructing the gaze point regression model based on the viewing distance, the relative distances and positions of the test points on the photographing device; calculating a probability of the gaze point position being mapped on a target region of the photographing device, comprising: acquiring the gaze point position in real time based on the randomly moving target region and the randomly moving non-target region; calculating a first number of times that the gaze point position is mapped on the target region; calculating a second number of times that the gaze point position is mapped on the non-target region; calculating a total number of times that the gaze point position is mapped on the photographing device according to the first number and the second number; calculating the probability of the gaze point being mapped on the target region of the photographing device based on the first number and the total number; determining a screening result according to the probability.

2. The screening method of claim 1, wherein, Before the acquiring of the face image by using the photographing device, the method further comprises: acquiring training data, verification data and a verification label corresponding to the verification data; inputting the training data into a neural network to obtain initial key point detection models corresponding to different hyperparameters according to a plurality of different hyperparameters set in advance; inputting the verification data into the initial key point detection models corresponding to different hyperparameters respectively to obtain initial key point detection results output by each initial key point detection model; calculating loss function values of each initial key point detection model according to each initial key point detection result and the verification label; comparing the loss function values of each initial key point detection model to determine a model with the minimum loss function value in the plurality of initial key point detection models as a trained key point detection model.

3. The screening method of claim 2, wherein, After the determination of the trained key point detection model, the method further comprises: acquiring test data and a test label corresponding to the test data; inputting the test data into the trained key point detection model to test and generate a test key point detection result; According to the test key point detection result and the test label, an error value and a fitting degree of the trained key point detection model are calculated; If the error value is greater than a first threshold or the fitting degree is less than a second threshold, the steps of obtaining training data and verification data are returned to be executed; If the error value is not greater than the first threshold and the fitting degree is not less than the second threshold, the trained key point detection model is taken as the preset key point detection model.

4. The screening method of claim 1, wherein, The pupil key points include a first pupil key point corresponding to a left eye and a second pupil key point corresponding to a right eye, and the eye corner key points include a left inner eye corner key point and a right inner eye corner key point. The relative distances of the pupil key points and the eye corner key points in the second target detection image are calculated, including: The distance between the first pupil key point and the left inner eye corner key point is taken as a relative distance corresponding to the left eye. The distance between the second pupil key point and the right inner eye corner key point is taken as a relative distance corresponding to the right eye.

5. The screening method of claim 1, wherein, According to the probability, a screening result is determined, including: According to the probability of the gaze point position being mapped on the target region of the shooting device, a risk score of autism of a person to be screened is determined, and the risk score is taken as the screening result.

6. A screening device characterized by, The screening device includes an obtaining module, a detection module, a mapping module and a screening module. The obtaining module is configured to obtain a face image by using a shooting device. A plurality of test points are arranged on the shooting device. A target region and a non-target region are arranged on the shooting device and move randomly. The detection module is configured to input the face image into a preset key point detection model to generate a first target detection image labeled with target key points. The mapping module is configured to input the first target detection image into a preset gaze point regression model to obtain a gaze point mapped on the shooting device and determine a gaze point position of the gaze point on the shooting device. The gaze point regression model is constructed, including: obtaining a viewing distance between a shooting object and the shooting device; performing image cropping on images of the shooting object viewing different test points to obtain a head image containing a head region of the shooting object; inputting the head image into the preset key point detection model to generate a second target detection image labeled with pupil key points and eye corner key points; calculating relative distances of the pupil key points and the eye corner key points in the second target detection image; and constructing the gaze point regression model based on the viewing distance, the relative distances and positions of the test points on the shooting device. The screening module is configured to calculate a probability of the gaze point position mapping on the target region of the photographing device, including: acquiring the gaze point position in real time based on the target region and the non-target region of the random movement; calculating a first number of times that the gaze point position maps on the target region; calculating a second number of times that the gaze point position maps on the non-target region; calculating a total number of times that the gaze point position maps on the photographing device according to the first number of times and the second number of times; calculating a probability of the gaze point mapping on the target region of the photographing device based on the first number of times and the total number of times; and determining a screening result according to the probability.

7. An electronic device, comprising: The electronic device includes a processor and a memory, and the processor is configured to implement the screening method of any one of claims 1 to 5 when executing the computer program stored in the memory.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is configured to implement the screening method of any one of claims 1 to 5 when executed by the processor.

Images