Automatic dominant eye and dominant brain judgment method and system based on image analysis
Through an automated method based on image analysis, deep neural networks are used to identify pupil locations and actions, the problems of strong subjectivity and low efficiency in the existing technology are solved, and efficient and accurate large-scale testing of the dominant eye and the dominant brain are achieved.
Patent Information
- Application Number
- CN202411179131.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-11-22
AI Technical Summary
The existing methods of testing dominant eye and dominant brain are highly subjective and difficult to apply to large-scale tests, especially for children and those with comprehension disorders, and cannot accurately identify visual differences in special groups, are inefficient and cannot meet school psychological education. Research and national psychological census needs.
Using an automated method based on image analysis, we use the subject's images, identify key images, and obtain monocular and facial attribute views. After fusion, we determine the correspondence between the dominant eye and the identity. We use deep neural network to identify the pupil position and movement to judge the dominant eye. Improve test accuracy and efficiency.
Large-scale dominant eye and dominant brain tests with high objectivity and low subjectivity are achieved, and recognition accuracy and efficiency are improved. They are suitable for large-scale sample surveys, especially for children and special populations.
Smart Images

Figure CN119006430A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of dominant eye and dominant brain determination, and in particular to an automated dominant eye and dominant brain determination method and system based on image analysis. Background Art
[0002] The determination of eye dominance is important in at least the following areas:
[0003] 1. Ophthalmological diagnosis: The dominant eye affects eye habits. By determining the dominant eye of the patient, the ophthalmologist can better understand the patient's visual habits and diagnose and correct binocular vision problems, such as strabismus or amblyopia. If the visual acuity of the two eyes is very different, it may affect the visual quality;
[0004] 2. Career choice and sports: Some professions (such as pilots, snipers, surgeons, etc.) and some sports that require precise visual positioning (such as shooting, tennis, baseball, etc.) require good monocular vision and dynamic or static stereoscopic vision, and the requirements for the dominant eye are even stronger;
[0005] 3. Education, learning and daily life: In school education, understanding students' dominant eye can help teachers adjust teaching methods and materials to suit the visual needs of different students; in daily life, such as driving, reading, using computers, etc., understanding and relying on the dominant eye can improve visual efficiency and comfort.
[0006] Furthermore, the connection between the dominant eye and the dominant brain lies in the optic chiasma. The information of the right visual field of the human eye enters the left cerebral visual cortex through the left optic tract (composed of the optic nerve on the temporal side of the left eye and the optic nerve on the nasal side of the right eye); the information of the left visual field enters the right cerebral visual cortex through the right optic tract (composed of the optic nerve on the temporal side of the right eye and the optic nerve on the nasal side of the left eye). In layman's terms, the left brain is mainly responsible for receiving the vision of the right eye, and the right brain is mainly responsible for receiving the vision of the left eye.
[0007] The development of the left and right brains is always unbalanced and asymmetrical, and there is a dominant brain phenomenon. The dominant brain phenomenon refers to the imbalance in the development of the two hemispheres of the human brain, that is, the human vision forms a double image on the left and right sides of the retina and the left and right brain visual centers of the cerebral hemispheres through binocular competition and binocular parallax. There must be a unilateral advantage phenomenon, that is, the dominant eye and dominant brain phenomenon, otherwise the human vision will see two different perspectives (for example, the human brain usually takes the dominant eye imaging as the leading factor, and combines the binocular double images into a single image; when the human head is hit, the brain is temporarily unable to process binocular information, and will see two non-overlapping perspectives). According to the optic nerve crossing phenomenon, the association between the dominant eye and the dominant brain is universal. For example, people with left eye dominance generally have right brain dominance, and people with right eye dominance generally have left brain dominance. In the academic field, the left brain is usually associated with functions such as logical reasoning, language processing, mathematical analysis, and attention to details; the right brain is related to functions such as creative thinking, imagination, art appreciation, music comprehension, and spatial perception. People who are right-brain dominant tend to be emotional, artistic, ancestral-brained, and have the Dionysian spirit; people who are left-brain dominant tend to be rational, modern, academic, and have the Apollo spirit.
[0008] On the one hand, brain science research shows that at least 60% to 80% of external information is obtained through vision. As the most important sense for humans, the left and right dominant eyes interact with the left and right dominant brains. During a person's life, the dominant eye and the dominant brain develop together, and are often affected by the outside world and undergo long-term or short-term changes. For example, emotional frustration, changes in the nature of work, brain damage and other reasons can all lead to chronic or instantaneous reversal of the dominant brain. Testing the dominant eye helps reflect the impact of the environment on the brain, which is also an important topic in psychology research on psychological trauma and psychological changes.
[0009] On the other hand, the study of the dominant brain not only enables us to understand the principles of brain functions such as cognition, thinking, consciousness and language, but also has great scientific significance for human self-understanding. By analyzing the neural basis of various brain functions, the study of the dominant brain has important clinical significance for the effective diagnosis and treatment of brain diseases. In addition, brain-like research inspired by brain science has also provided new ideas and methods for the development of a new generation of artificial intelligence and new information industries, promoting innovation and development in these fields. Determining the dominant brain is of great significance for psychological research and treatment.
[0010] Therefore, by testing the dominant eye and the dominant brain, professionals in this field can obtain a huge amount of information for research or statistics.
[0011] However, existing cutting-edge technologies for testing dominant eye have the following defects:
[0012] First, the existing methods for determining the dominant eye or dominant brain are based on a subjective perspective and the test results are obtained based on the test subject's language feedback. This places high demands on the test subject's comprehension ability and requires the test subject to accurately understand the purpose of the test and accurately perceive the visual difference between the dominant eye and the accessory eye. It also requires the test subject to accurately describe the perceived binocular differences through language. Therefore, when testing special groups such as children and people with comprehension disabilities, the accuracy of the feedback obtained cannot be guaranteed. Psychological research on special groups is often an important topic in psychology.
[0013] Second, the existing methods cannot obtain samples on a large scale, and the test methods can only be used for small-scale or one-on-one measurements, which cannot be applied to school psychological education research, national psychological census and other fields. The reason is that the test results can only be recorded by the test subjects through personal feedback of the test results. For large-scale surveys, it is necessary to visit the test subjects one by one and explain the test methods repeatedly. The test results are inefficient, inconvenient, and not statistically significant.
[0014] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention
[0015] The purpose of the present invention is to provide an automated dominant eye and dominant brain determination method based on image analysis, which can solve the problems existing in the background technology.
[0016] To achieve the above object, the present invention provides an automated dominant eye and dominant brain determination method based on image analysis, which is applied to an automated dominant eye and dominant brain determination system based on image analysis, and is characterized in that it comprises the following steps:
[0017] S100: Acquire the image of the subject;
[0018] S200: identifying key images based on the subject image;
[0019] S300: obtaining a monocular attribute view based on the key image;
[0020] S400: Obtaining a facial attribute view based on the key image;
[0021] S500: fusing the monocular attribute view and the facial attribute view to obtain a first identity correspondence relationship;
[0022] S600: Based on the subject image, identify the subject's action and obtain a dominant eye discrimination result;
[0023] S700: integrating the dominant eye discrimination result and the first identity correspondence to determine a second identity correspondence;
[0024] S800: Summarizing the second identity correspondence relationship to generate a dominant eye and identity correspondence result;
[0025] The subject images refer to a collection of multiple pictures covering the subject, and the pictures of the subject images are arranged in order.
[0026] In one or more implementations, step S200 includes:
[0027] S210: Identify the pupil position of the subject in the subject image;
[0028] S220: traversing the subject's image, determining the change in the number of the subject's pupils based on the subject's pupil positions, and obtaining the key image;
[0029] S230: Recording time information of the key picture;
[0030] The key image refers to the image at the moment when the subject's pupil position is blocked.
[0031] In one or more implementations, step S300 includes:
[0032] S310: performing image recognition on the key image to obtain monocular attribute confidences, and sorting the monocular attribute confidences to obtain sorting results;
[0033] S320: Obtaining monocular attributes based on the sorting result;
[0034] S330: Based on the monocular attribute, modify the monocular attribute view;
[0035] The single eye attribute includes the left eye or the right eye.
[0036] In one or more implementations, step S310 includes:
[0037] S311: Identifying monocular features based on the pupil position;
[0038] S312: Obtaining a monocular feature confidence level according to the monocular feature;
[0039] S313: weighting the monocular feature confidence to obtain the monocular attribute confidence;
[0040] Among them, the monocular features include: tear gland position, eye canthus height, eye frame contour, and eye structure.
[0041] In one or more implementations, step S400 includes:
[0042] S410: Identify the key image, and establish a facial feature network by splitting facial features of the key image;
[0043] S420: performing intelligent processing on the facial feature network to obtain facial attribute confidence;
[0044] S430: Obtain a facial attribute view according to the facial attribute confidence.
[0045] In one or more implementations, step S420 includes:
[0046] S421: Identify the occluder, determine the possibility that the occluder is the target object according to the similarity between the occluder and the target object, and obtain the occluder identification result;
[0047] S422: According to the result of the occlusion identification, the intelligent processing method is selected to obtain the facial attribute confidence;
[0048] Wherein, the step S422 includes the following steps:
[0049] S421: Importing and recording the subject's identity data, wherein the identity data includes a facial picture;
[0050] S422: Identify the occluder, and determine the possibility that the occluder is the target object according to the similarity between the occluder and the target object, and obtain the occluder identification result;
[0051] S423: Obtaining a threshold value, and selecting the optimal intelligent processing method according to the result of the occlusion identification to obtain the facial attribute confidence;
[0052] Wherein, the step S423 includes the following steps:
[0053] When the occluder identification result is that the occluder is a target object, the intelligent processing method is: identifying the facial feature network through a convolutional neural network and comparing it with the preprocessing result to obtain the facial attribute confidence, and ending step S423;
[0054] When the occluder identification result is that the occluder is not the target object, determining the destruction rate of the facial feature being occluded;
[0055] When the destruction rate is lower than the threshold, the intelligent processing method is: introducing an attention mechanism, ignoring the obscured facial features, identifying the facial feature network and weighting it to obtain a score, obtaining the facial attribute confidence according to the score, and ending step S423;
[0056] When the destruction rate is not lower than the threshold, the intelligent processing method is: traversing the imported subject identity data, using the subject identity data as material, reconstructing the blocked part, and generating multiple virtual character portraits; identifying the similarity between the virtual character portrait and the subject identity data, and obtaining the facial attribute confidence;
[0057] Among them, the occluder refers to a prop used to cover the pupil and conduct a dominant eye test; the target object refers to a designated occluder, and the characteristic information of the target object is stored in an automated dominant eye and dominant brain determination system based on image analysis; the preprocessing result refers to the preprocessing result obtained by superimposing the characteristics of the target object at the pupil position of the facial image in advance to form a preset occluded image.
[0058] In one or more implementations, step S500 includes:
[0059] S510: Retrieve key images;
[0060] S520: Based on the key image, retrieve the pupil position of the subject;
[0061] S530: Based on the subject's pupil position, fuse the monocular attribute view and the facial attribute view to obtain a first identity correspondence relationship;
[0062] In one or more implementations, step S530 includes:
[0063] S531: Based on the subject's pupil position, identifying close monocular positions, and obtaining a closeness ranking of the monocular positions;
[0064] S532: Based on the monocular position proximity sorting, the monocular attribute views are merged, the wrong monocular position is eliminated, and the monocular position proximity sorting is corrected;
[0065] S533: sorting based on the proximity of the monocular positions, fusing the facial attribute views, and determining the monocular position inclusion based on the facial feature network;
[0066] S534: Based on the monocular position including the degree, associate the monocular attribute with the identity data to obtain a first identity correspondence.
[0067] In one or more implementations, step S600 includes:
[0068] S610: Taking the time information of the key image as a starting point, traversing the subject's image to generate a monocular action atlas;
[0069] S620: Based on the monocular action atlas, recognize the action and determine the monocular attribute;
[0070] S630: Obtaining a dominant eye discrimination result based on the monocular attribute;
[0071] The action refers to the conditioned reflex action of the contralateral eyeball when one pupil is blocked.
[0072] In a second aspect, the present invention provides an automated dominant eye and dominant brain training method based on image analysis, based on the same concept as the automated dominant eye and dominant brain determination method based on image analysis, the automated dominant eye and dominant brain training method based on image analysis is used to train the method for identifying the subject's actions in step S600, comprising:
[0073] Obtain a single occluded image of the subject;
[0074] Based on the subject's occluded single image, obtaining the subject's identity information;
[0075] Based on the single image blocked by the subject, obtaining the dominant eye information of the subject;
[0076] Associating the dominant eye information with the subject's identity information to determine the dominant eye;
[0077] The determined dominant eye is used as corpus to train the automated dominant eye and dominant brain determination method based on image analysis, thereby reducing the loss of the dominant eye identity corresponding result relative to the determined dominant eye.
[0078] Compared with the prior art, the multiple technical solutions and embodiments provided by the present invention have at least the following technical effects or advantages:
[0079] 1. The test images of the subjects are captured by video equipment, which is objective and avoids the influence of subjectivity on the test results. It can also be used for large-scale testing.
[0080] 2. By determining the occlusion time through the pupil position, it is possible to determine the time when the evidence that can prove the dominant eye is located, exclude invalid evidence, and improve the accuracy of the test.
[0081] 3. Identify the attributes of a single eye through the structure of the human eye. For example, the probability of judging whether the identified eye is a left eye or a right eye can be determined by the tear gland being located on the left or right side. This can improve recognition accuracy. Finding key features can also improve recognition efficiency.
[0082] 4. Because some facial features of the subjects are blocked when testing the dominant eye, the facial attribute confidence is obtained through intelligent processing, which can maximize the optimization of the algorithm and provide a more accurate and efficient way to identify the subject's identity.
[0083] 5. By identifying the conditioned reflex actions of the subjects when their pupils are blocked under natural circumstances, determine whether the blocked eye is the dominant eye. By training the algorithm, compared with the existing technology, the conditioned reflex serves as the most objective and reliable evidence, providing the evidence with the greatest accuracy in dominant eye judgment. BRIEF DESCRIPTION OF THE DRAWINGS
[0084] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and can be considered as illustrative of various combinations of preferred embodiments and do not constitute an improper limitation on the present invention. In the drawings:
[0085] Figure 1 A schematic diagram of the overall process of an automated dominant eye and dominant brain determination method based on image analysis provided by the present invention;
[0086] Figure 2 A schematic diagram of the dominant eye test steps provided by the present invention;
[0087] Figure 3 A schematic flow chart of step S200 of an automated dominant eye and dominant brain determination method based on image analysis provided by the present invention;
[0088] Figure 4 A schematic flow chart of step S300 of an automated dominant eye and dominant brain determination method based on image analysis provided by the present invention;
[0089] Figure 5 A schematic diagram of a preferred flow chart of step S310 of an automated dominant eye and dominant brain determination method based on image analysis provided by the present invention;
[0090] Figure 6 The overall flow diagram of step S400 of an automated dominant eye and dominant brain determination method based on image analysis provided by the present invention;
[0091] Figure 7 The overall flow diagram of step S500 of an automated dominant eye and dominant brain determination method based on image analysis provided by the present invention;
[0092] Figure 8 A schematic diagram of the association between pupil position and dominant eye information in an automated dominant eye and dominant brain determination method based on image analysis provided by the present invention;
[0093] Fig. 9 Schematic diagram of the optic chiasm and sympathetic nerves. DETAILED DESCRIPTION
[0094] Unless explicitly stated otherwise, throughout the specification and claims, the term "comprise" or its variations such as "include" or "comprising", "made of", etc. will be understood to include the stated elements or components but not to exclude other elements or other components.
[0095] The purpose of the present invention is to provide an automated dominant eye and dominant brain determination method and system based on image analysis, so as to achieve the effects of improving the accuracy of testing dominant eye, providing large-scale statistical dominant eye adaptation scenarios and improving their efficiency.
[0096] Embodiment 1:
[0097] This embodiment provides an automated dominant eye and dominant brain determination method based on image analysis, which is applied to an automated dominant eye and dominant brain determination system based on image analysis, and is characterized in that it includes the following steps:
[0098] S100: Acquire the image of the subject;
[0099] Existing technology uses the method of collecting electrical signals or roughly measuring visual acuity differences to simply determine the dominant eye. The experiments are not rigorous and the data reliability is questionable.
[0100] This application proposes a method of artificial dominant eye testing, such as Figure 2 As shown, its content includes that the subject's eyes look at the target, the subject moves the object and visually overlaps the object with the target to obtain the dominant eye result; its principle is that visual imaging is mainly based on the dominant eye, and the auxiliary eye imaging has a lower weight in vision. When the object blocks the correct dominant eye pupil, the vision on the dominant eye side will see the object overlap with the target, and the vision on the auxiliary eye side will see the target at the gaze position. At the same time, the object deviates from the target position. Therefore, when the brain processes the vision of the two, based on the dominant eye, the subject will think that the object overlaps with the target; therefore, when the subject overlaps the object with the target, it can be judged that the dominant eye is the blocked side. However, this method still has the problem of manual testing as described in the background technology, so there are still technical problems.
[0101] Furthermore, in order to improve the accuracy of the test and facilitate the acquisition of large-scale samples for research, the present application obtains the subject's image through a dominant eye test step, and the dominant eye test step includes:
[0102] 1. A camera is set up in the test area. Preferably, the camera is an ultra-high-definition camera, the purpose of which is to provide a resolution capable of identifying the pupil;
[0103] 2. The test subject enters the test area and is informed of the test process. Preferably, the experimental test steps adopt a single-blind test method, which means that the test subject is not informed of the purpose of the test in advance, so that the test subject is in the most natural state, improves the reaction speed, and shows the conditioned reflex more naturally;
[0104] 3. The subject follows the test process, looks at the camera device, and uses an obstruction to cover the left and right eyes respectively, and keeps looking at the camera device during this period;
[0105] 4. Obtain the subject’s image.
[0106] Among them, the subject image refers to a collection of multiple pictures covering the subject, including but not limited to multiple pictures or video recordings taken continuously, preferably in the form of video recordings, and the multiple pictures of the subject image are arranged continuously in order from early to late according to the time of shooting.
[0107] This embodiment adopts a deep neural network (DNN) method to perform image analysis, and proposes a technical solution of setting a camera device at the target to realize the method of automatic dominant eye and dominant brain determination based on image analysis. The advantage is that the camera device captures the image of the subject in the dominant eye test step, and performs data processing to obtain the dominant eye. There is objectivity, and subjective intervention is impossible, thus avoiding the influence of subjectivity on the test results. Therefore, the setting of the camera device is essentially based on the consideration of changing the test perspective, turning the passive test into an active test; further, the existing implementation methods are based on the human eye as the main perspective, and the technicians in this field cannot obtain a breakthrough point based on the technical inspiration provided by the existing technology; and the technicians in this field cannot find the problems existing in the existing technology, and cannot obtain the motivation for improvement based on the technical problems; further, the technicians in this field cannot know the technical effect brought by the camera device as the main perspective before the present technical solution, so the improvement of the technology is blind, and even blindly piecing together the existing technology for a limited number of times cannot realize the present technical solution.
[0108] S200: identifying key images based on the subject image;
[0109] As a preferred implementation of this embodiment, step S200 includes:
[0110] S210: Identify the pupil position of the subject in the subject image;
[0111] S220: traversing the subject's image, determining the change in the number of the subject's pupils based on the subject's pupil positions, and obtaining the key image;
[0112] S230: Recording time information of the key picture;
[0113] Specifically, in order to confirm the precise time when the subject blocks his / her sight, the present embodiment adopts the method of identifying the pupil to determine. When implementing step S200, the subject's pupil position in the subject's image is first identified and marked by a deep neural network (DNN), and the subject's pupil position in the subject's image is traversed to determine the image at the moment when the subject's pupil position is blocked as the key image. Preferably, the camera device uses an ultra-high-definition camera to achieve clear pupil visibility, identify the pupil position through image recognition, and determine the key image and key frame based on the change in the number of pupils, so as to facilitate the subsequent search for evidence to prove the dominant eye based on the key image position.
[0114] The advantage of using the pupil position to determine the occlusion is that, in the process of implementing this embodiment, it is concluded that the evidence that can determine the dominant eye only exists for a very short time, resulting in extremely high requirements for the capture accuracy of the dominant eye evidence. Therefore, how to locate the interval where the dominant eye evidence may exist and exclude the interval where invalid evidence may exist is a technical problem that cannot be solved by the prior art. Based on the scene of dominant eye judgment, this embodiment implements a method of determining the interval where the dominant eye evidence exists without doubt by identifying the pupil position; since the visual imaging of the human eye passes through the pupil, when the pupil, the camera device and the occluding object form a line, it can be determined that the human eye vision is blocked, and the dominant eye is judged based on uncontrolled conditioned reflexes. Therefore, by identifying the change in the number of pupils through a deep neural network (DNN), the correct interval where the dominant eye evidence exists can be determined. Since the human eye has unconscious activities when it is relaxed, the prior art cannot judge the dominant eye based on the correct time interval, and will calculate the unconscious eye activity, which affects the dominant eye judgment and obtains incorrect dominant eye information. This embodiment can obtain substantial evidence to determine the dominant eye, and the test accuracy is significantly improved.
[0115] Furthermore, when pupil occlusion occurs, the key image is identified. Pupil occlusion of different subjects may occur at different times, so multiple key images are captured and processed separately to obtain information data of different subjects.
[0116] Specifically, the specific method of constructing a deep neural network (DNN) model is as follows:
[0117] Use ultra-high-definition camera equipment to capture continuous images or videos of the subjects during the test to ensure that the pupils and facial features can be clearly captured; and segment the continuous images or videos into single-frame images, and perform preprocessing operations such as grayscale, denoising, and normalization on the images to improve image quality and facilitate subsequent processing. The architecture of the deep neural network (DNN) model includes input layer, convolution layer, fully connected layer, output layer, loss function, and optimization algorithm. The input layer is used to receive preprocessed image data. The convolution layer is used to automatically extract features in the image, such as edges, textures, etc., using multiple convolution layers. Each convolution layer is usually followed by an activation function (such as ReLU) and a pooling layer (such as maximum pooling) to reduce the amount of calculation and prevent overfitting. The fully connected layer is used to flatten the feature map output by the convolution layer and perform classification or regression through the fully connected layer. The output layer is used to output corresponding results according to specific tasks, such as pupil position coordinates, monocular attributes (left eye / right eye), facial attribute confidence, etc. The loss function is used to select the appropriate loss function according to the task, such as mean square error (MSE) for regression tasks and cross-entropy loss (Cross-Entropy Loss) for classification tasks. The optimization algorithm is used to update the network weights and minimize the loss function using optimization algorithms such as Adam and SGD. Training a deep neural network model: Prepare a large amount of labeled image data, including labels such as pupil position, monocular attributes (left eye / right eye), and facial features. Divide the labeled data set into a training set, a validation set, and a test set. Use the training set data to train the DNN model, calculate the loss through forward propagation, and update the network weights through back propagation. Use the validation set regularly to evaluate the model performance, adjust the hyperparameters, and prevent overfitting. After training is completed, use the test set to evaluate the generalization ability of the model. Apply a deep neural network model: Pupil position recognition: Input the preprocessed image into the DNN model; the model outputs the coordinates of the pupil position. Monocular attribute recognition: Based on the pupil position, the eye area image around the pupil is cropped; the eye area image is input into another DNN model (or another part of the same model) to identify the monocular attributes (left eye / right eye). Facial attribute recognition: Facial features are extracted from the entire image, and the facial attribute confidence is output through the DNN model; combined with the occlusion discrimination results, different intelligent processing methods (such as attention mechanism, virtual character portrait reconstruction, etc.) are used to improve the recognition accuracy. Fusion of monocular attributes and facial attributes: Based on the pupil position and monocular attributes, a monocular attribute view is constructed; based on the facial attribute confidence, a facial attribute view is constructed; the two views are fused to obtain the identity correspondence. Dominant eye discrimination: Analyze the subject's actions when the pupil is occluded, such as blinking, pupil focus changes, etc.; combine the monocular attributes and action recognition results to comprehensively judge the dominant eye. Performance evaluation: Evaluate the model's accuracy, recall rate, F1 score and other indicators through actual test data. Model optimization: Adjust the network structure, hyperparameters, or increase the amount of data based on the evaluation results to further improve model performance.Through the above steps, a deep neural network model for image analysis can be constructed.
[0118] S300: obtaining a monocular attribute view based on the key image;
[0119] As a preferred implementation of this embodiment, step S300 includes:
[0120] S310: performing image recognition on the key image to obtain monocular attribute confidences, and sorting the monocular attribute confidences to obtain sorting results;
[0121] Specifically, the monocular attribute includes the left eye or the right eye; the monocular attribute view refers to a comprehensive view that associates each monocular attribute with the pupil position. In the subject image, the pupil position changes over time, so the monocular attribute view is a view that dynamically reflects the relationship between the pupil and the monocular attribute. For example, when the subject is one person, after identification by this embodiment, the monocular attribute result corresponding to one pupil is obtained, and it remains consistent in time, that is, the pupil position changes in the subject image, but the monocular attribute corresponding to a single pupil is determined and unchanged, providing a way to dynamically determine the monocular attribute; the purpose of setting the monocular attribute view is that when testing the dominant eye on a large scale, such as testing a class of about 30 to 40 people, each subject blocks the pupil at a different time, and the time when the dominant eye evidence appears is different. Therefore, due to complex factors, it is impossible to simply obtain the dominant eye discrimination result of the subject at the same time, and the monocular attribute view can update and correct the monocular attributes of each subject when the pupil is blocked at different times, and finally form a unified and dynamic monocular attribute set, which is convenient for further judgment; therefore, this embodiment can still obtain unified monocular attribute results even for large-scale testing.
[0122] As a preferred implementation of this embodiment, step S310 includes:
[0123] S311: Identifying monocular features based on the pupil position;
[0124] S312: Obtaining a monocular feature confidence level according to the monocular feature;
[0125] S313: weighting the monocular feature confidence to obtain the monocular attribute confidence;
[0126] Specifically, in order to identify the left and right eyes through a deep neural network (DNN) when the face is partially occluded, the present embodiment provides a method for identifying eye features through coordinate information of the pupil position, and the step S311 includes: determining the occluded pupil position; according to the occluded pupil position, searching for the pupil position of a picture with an earlier picture shooting time sequence, preferably, searching for the pupil position of the picture with the earliest shooting time, because the pupil position is not blocked, the eye features can be identified; because the facial features are destroyed by occlusion, it is difficult to confirm the correspondence between the two eyes when there are many subjects. For example, when the subjects are of similar height and stand close to each other, the distance between multiple monocular eyes is close, so finding the corresponding eye of the occluded eye requires recognition and analysis of the face, while the method of judging whether the monocular attribute of the occluded eye itself is the left eye or the right eye only needs to determine the change of the pupil position over time, which consumes less computing power.
[0127] Further, according to the structure of the human eye, from the pupil outward, there are the iris, sclera, and eyelid in order. Step S311 is based on the coordinates provided by the pupil position, by finding similar coordinates and identifying monocular features, for example, identifying the iris outside the pupil, identifying the outer sclera position according to the iris position, identifying the eyelid position according to the sclera position, identifying the position between the eyelid and the sclera, and determining the position of the tear gland.
[0128] Step S312 obtains the monocular feature confidence by identifying the monocular feature, which means arranging the similarities of the monocular features to obtain the monocular feature confidence covering a set of multiple monocular feature similarities; and in step S313, the monocular feature similarities in the weighted monocular feature confidence are weighted to obtain the confidence that the monocular is the left eye or the right eye, specifically, the monocular attribute probability corresponding to the pupil is obtained by weighting and according to the comprehensive score, and the monocular attribute confidence is sorted, and the monocular attribute with the highest confidence is selected as the sorting result; for example, the lacrimal gland feature is identified, and the lacrimal gland is located to the left of the pupil. On the other hand, due to the structure of the two eyes, the positions of the tear glands of the two eyes are opposite, so the position of the tear glands is used as evidence with a higher weight to prove the attribute of the single eye. After the tear gland features are weighted, the confidence that the attribute of the single eye is the left eye is greatly improved; therefore, the determination of the attribute of the single eye in this embodiment is based on image recognition based on the structure of the eyeball, and the basis for distinguishing the left and right eyes is sought according to the difference between the left and right eyes; through this embodiment, it is possible to determine whether the occluded eye is the left eye or the right eye through the features of the single eye without the need for complex face recognition. In the subsequent steps of this embodiment, the attribute of the single eye can be associated with other forms.
[0129] Among them, the monocular features include: tear gland position, eye canthus height, eye frame contour, and eye structure.
[0130] S320: Obtaining monocular attributes based on the sorting result;
[0131] Specifically, the confidence of the monocular attribute being the left eye and the confidence of the monocular attribute being the right eye are continuously weighted and sorted, and the monocular attribute with a higher confidence is selected as the monocular attribute.
[0132] S330: Based on the monocular attribute, modify the monocular attribute view;
[0133] Specifically, in order to obtain a unified monocular attribute view when multiple subjects have different occlusion times and reaction times, when individual subjects occlude their pupils, key pictures are captured and identified and recorded. Further, when the monocular attributes are identified and obtained, the monocular attributes are added to the monocular attribute view. Before step S300, the monocular attribute view only includes the pupil position as information, and the pupil position information is blank. After traversing the subject images, the monocular attribute view is filled with all monocular attributes by the system and associated with the pupil position, forming a coordinate system in which the monocular attributes and the pupil position are variables in a one-to-one correspondence.
[0134] S400: Obtaining a facial attribute view based on the key image;
[0135] Specifically, since the subject's image is partially occluded and the facial features are destroyed, this embodiment uses a step-by-step method to determine the monocular attributes and recognize the face, and a comprehensive judgment method is used, wherein the facial attribute view is a comprehensive view that associates the pupil position with the facial attributes, and the facial attributes refer to the identity of the subject.
[0136] As a preferred implementation of this embodiment, step S400 includes:
[0137] S410: Identify the key image, and establish a facial feature network by splitting facial features of the key image;
[0138] S420: performing intelligent processing on the facial feature network to obtain facial attribute confidence;
[0139] S430: Obtain a facial attribute view according to the facial attribute confidence.
[0140] Specifically, the facial features contained in the key image are split according to type; the purpose of identifying and splitting the facial features is that some facial features may be destroyed, for example, the face shape may be unclear due to occlusion, the facial features are split and a facial feature network is established to obtain facial attribute confidence, which refers to the degree of similarity between facial attributes and subject data.
[0141] The facial features include but are not limited to: face shape, hair color, nose shape, etc. The facial feature network refers to a collection of various facial features of all subjects.
[0142] As a more preferred implementation of this embodiment, step S420 includes:
[0143] S421: Importing and recording the subject's identity data, wherein the identity data includes a facial picture;
[0144] S422: Identify the occluder, and determine the possibility that the occluder is the target object according to the similarity between the occluder and the target object, and obtain the occluder identification result;
[0145] S423: Obtaining a threshold value, and selecting the optimal intelligent processing method according to the result of the occlusion identification to obtain the facial attribute confidence;
[0146] Specifically, the system imports database data covering the identity data of the subject and records it into the storage area of the system; the target object refers to a preferred obstruction with easily identifiable features, such as a stick of bright colors, etc. Preferably, the target object is provided when the subject enters the test area as an optional, preferred but non-mandatory accelerated identification method. When the system identifies the obstruction as a target, it determines that the obstruction identification result is that the obstruction is a target; when the system identifies that the obstruction is not a target, it determines that the obstruction identification result is that the obstruction is not a target. The target object is used to improve the recognition of the subject's image and save computing power for intelligent processing for large-scale testing; the preferred intelligent processing method includes comparing the obstruction identification result with a comparison threshold.
[0147] Wherein, the step S423 includes the following steps:
[0148] When the occluder identification result is that the occluder is a target object, the intelligent processing method is: identifying the facial feature network through a convolutional neural network and comparing it with the preprocessing result to obtain the facial attribute confidence, and ending step S423;
[0149] When the occluder identification result is that the occluder is not the target object, determining the destruction rate of the facial feature being occluded;
[0150] When the destruction rate is lower than the threshold, the intelligent processing method is: introducing an attention mechanism, ignoring the obscured facial features, identifying the facial feature network and weighting it to obtain a score, obtaining the facial attribute confidence according to the score, and ending step S423;
[0151] When the destruction rate is not lower than the threshold, the intelligent processing method is: traversing the imported subject identity data, using the subject identity data as material, reconstructing the blocked part, and generating multiple virtual character portraits; identifying the similarity between the virtual character portrait and the subject identity data, and obtaining the facial attribute confidence;
[0152] Among them, the occluder refers to a prop used to cover the pupil and conduct a dominant eye test; the target object refers to a designated occluder, and the characteristic information of the target object is stored in an automated dominant eye and dominant brain determination system based on image analysis; the preprocessing result refers to the preprocessing result obtained by superimposing the characteristics of the target object at the pupil position of the facial image in advance to form a preset occluded image.
[0153] Specifically, as a selective implementation of this preferred embodiment, based on the imported and collected subject identity data, the characteristic image of the target object is superimposed at the pupil position of the facial image to form a preset occlusion image set. When the occlusion identification result is that the occlusion is the target object, the preset occlusion image set is called, and based on the preset occlusion image set, the convolutional neural network is used to identify and compare the facial feature network to determine the similarity of the subject's identity and obtain the facial attribute confidence.
[0154] When the result of the occlusion discrimination is that the occlusion is not the target object, the destruction rate of the facial features being occluded is determined. The destruction rate refers to the amount of loss of unrecognized facial features, and the unrecognized facial features are lost by weighting the number of unrecognized facial features. The value of the threshold is imported by the tester or preset by the system. It can be understood that the threshold changes according to the algorithm requirements, and the specific value can be confirmed according to the guidance of technicians in this field, which will not be described in detail here. The system obtains the value of the threshold by presetting and / or receiving input signals, and selects the optimal intelligent processing method by further comparing the destruction rate with the threshold.
[0155] As a selective implementation of this preferred embodiment, when the destruction rate is lower than the threshold, the intelligent processing method is: introducing an attention mechanism, the attention mechanism focuses the system's attention on the unobstructed facial features, rather than treating all facial features equally. In other words, the attention mechanism ignores the obstructed parts and focuses on the unobstructed parts. When the destruction rate is lower than the threshold, the subject's image still retains a large amount of facial feature information, the attention mechanism is used to identify the focus, ignore the obstructed facial features, identify the significant features with a low degree of destruction, and weight them to obtain the face similarity score, and obtain the facial attribute confidence according to the score.
[0156] As an optional implementation method of this preferred embodiment, when the destruction rate is not lower than the threshold value and the degree of facial occlusion and destruction is large, the intelligent processing method is: traverse the imported subject identity data, use the subject identity data as material, reconstruct the occluded part, and generate multiple virtual character portraits; further, in order to reconstruct the occluded part, first determine the unobstructed part of the facial features, search for all facial features, determine the lost facial features, and reconstruct the lost part based on the corresponding facial features of the subject identity data to generate a reconstructed part, replace the corresponding position of the original subject image with the reconstructed part, and generate a virtual character portrait. That is to say, by reconstructing the missing part of the facial features, this implementation method can generate an image not obstructed by the occluder from the image obstructed by the occluder, thereby achieving the effect of erasing the occluder. A plurality of virtual character portraits using identity data of a plurality of subjects are established, and the facial attribute confidence is obtained by identifying the similarity between the virtual character portrait and the original identity data of the subject; the method of establishing the virtual character portrait is strong in robustness, and the identity information of the subject can still be obtained by using a convolutional neural network (CNN) to identify the subject even if the reconstructed part has misaligned facial features in the overall picture.
[0157] Among them, the step S300 searches for the pupil position of the earliest picture taken to obtain a monocular attribute view; and the step S400 obtains the facial attribute view through the current picture. The difference is that the association between the monocular attribute and the pupil position only needs to be determined by identifying a single picture, while the facial attribute view requires the system to continuously identify the facial attributes of the subject. This embodiment determines the pupil position by traversing the subject's image, and links each information with the pupil position respectively, and then uses the pupil position to run through each attribute relationship to achieve the association of identity relationships.
[0158] Specifically, the method of constructing a convolutional neural network (CNN) is as follows:
[0159] Collect a large amount of facial image data from subjects, including images from different angles and lighting conditions, to ensure the diversity and comprehensiveness of the data, and annotate the collected facial images, mark the location and attributes of key facial features (such as eyes, nose, mouth, etc.), and perform preprocessing operations such as cropping, scaling, and normalization on the images so that the input images have a uniform size and pixel range. Design a CNN model architecture, including input layer, convolution layer, fully connected layer, and output layer. Input layer: accepts preprocessed facial images as input. Convolutional layers are used to automatically extract features from images using multiple convolutional layers. Each convolutional layer is usually followed by an activation function (such as ReLU) to increase nonlinearity, and a pooling layer (such as max pooling) to reduce the dimension and computation of the feature map. Example structure: Conv1: 32 3x3 convolution kernels, stride 1, padding 1, activation function ReLU. Pool1: 2x2 max pooling, stride 2. Conv2: 64 3x3 convolution kernels, stride 1, padding 1, activation function is ReLU. Pool2: 2x2 max pooling, stride 2. (You can add more convolutional layers and pooling layers as needed). Fully connected layers are used to flatten the feature maps output by the convolutional layer and then perform classification or regression through fully connected layers. Activation functions are usually included between fully connected layers. Example structure: FC1: 128 neurons, activation function is ReLU. Dropout: The dropout rate is 0.5 to prevent overfitting. FC2: Output layer, the number of neurons depends on the specific task (for example, in the face recognition task, the number of neurons in the output layer is equal to the number of subjects). The output layer is used to design the output layer according to the task requirements. For the face recognition task, the output layer usually uses the softmax function for multi-classification. Training CNN model: Choose an appropriate loss function, such as the cross entropy loss function, for multi-classification tasks. Optimization algorithm: Use optimization algorithms such as Adam and SGD to update the network weights and minimize the loss function. Training process: The labeled facial image data is divided into training set, validation set and test set. Use the training set data to train the CNN model, calculate the loss through forward propagation, and update the network weights through back propagation. After each epoch, use the validation set to evaluate the model performance, adjust the hyperparameters, and prevent overfitting. After the training is completed, use the test set to evaluate the generalization ability of the model. Apply the CNN model: Facial feature recognition: The preprocessed facial image is input into the trained CNN model, and the model outputs the confidence or category label of the facial feature. Feature comparison: In step S420, the facial features identified by the CNN model are compared with the preprocessing results (i.e., the facial image with the target object features superimposed) to calculate the similarity or confidence. Comprehensive judgment: Combined with other intelligent processing methods (such as attention mechanism, virtual character portrait reconstruction, etc.), comprehensively judge the confidence of facial attributes, etc. Finally: Evaluate the accuracy, recall rate, F1 score and other indicators of the CNN model through actual test data.And according to the evaluation results, adjust the network structure, hyperparameters or increase the amount of data to further improve the model.
[0160] S500: fusing the monocular attribute view and the facial attribute view to obtain a first identity correspondence relationship;
[0161] As a preferred implementation of this embodiment, step S500 includes:
[0162] S510: Retrieve key images;
[0163] S520: Based on the key image, retrieve the pupil position of the subject;
[0164] S530: Based on the subject's pupil position, fuse the monocular attribute view and the facial attribute view to obtain a first identity correspondence relationship;
[0165] Specifically, in order to determine the first identity correspondence, the monocular attributes and facial attributes are associated through the pupil position of the key image; for example, the occluded eye of the monocular attribute is the left eye, and the person identified by the corresponding facial attribute is A. It can be determined that subject A occludes the left eye in the key image, and the first identity correspondence is obtained. According to the first identity correspondence, it is convenient to determine whether the occluded eye of subject A is the dominant eye in the subsequent steps.
[0166] As a preferred implementation of this embodiment, step S530 includes:
[0167] S531: Based on the subject's pupil position, identifying close monocular positions, and obtaining a closeness ranking of the monocular positions;
[0168] S532: Based on the monocular position proximity sorting, the monocular attribute views are merged, the wrong monocular position is eliminated, and the monocular position proximity sorting is corrected;
[0169] S533: sorting based on the proximity of the monocular positions, fusing the facial attribute views, and determining the monocular position inclusion based on the facial feature network;
[0170] S534: Associating the monocular attributes with the identity data based on the monocular position including the degree, to obtain a first identity correspondence relationship;
[0171] The single eye position refers to the unobstructed eye position in the key image.
[0172] Specifically, in order to determine the corresponding monocular attributes, in step S531, the close monocular positions are identified in the key image. The closeness refers to the close position of the monocular position and the pupil position at the moment of occlusion in the key image. When the positions of multiple subjects are close and the distances between multiple pupils are close, it will be difficult to determine the correspondence between the subjects' binoculars. First, determine the proximity sorting of the monocular positions, call the monocular attribute view, exclude some monocular positions, sort the monocular positions by proximity, select, for example, the occluded pupil is a left eye, exclude all monocular positions with monocular attributes as the left eye, based on the corrected monocular position proximity sorting, call the facial attribute view, and obtain the monocular attributes according to the facial feature network, and determine the inclusion degree of the monocular position; the inclusion degree refers to the degree to which the monocular position and the pupil position are included in the facial features of the same subject, for example, the monocular position and the pupil position are both included in the same facial contour, and it is determined that the monocular position inclusion degree is high.
[0173] S600: Based on the subject image, identify the subject's action and obtain a dominant eye discrimination result;
[0174] As a preferred implementation of this embodiment, step S600 includes:
[0175] S610: Taking the time information of the key image as a starting point, traversing the subject's image to generate a monocular action atlas;
[0176] S620: Based on the monocular action atlas, recognize the action and determine the monocular attribute;
[0177] S630: Obtaining a dominant eye discrimination result based on the monocular attribute;
[0178] The action refers to the conditioned reflex action of the contralateral eyeball when one pupil is blocked.
[0179] Specifically, in order to determine the credibility of the occluded eye as the dominant eye, the dominant eye is identified by identifying the abnormal movement of a single eye. For example, when the dominant eye is occluded, the subject shows conditioned reflex movements such as more frequent blinking and more significant pupil focus changes compared to the occluded accessory eye. The dominant eye is identified based on the movement, which can increase the accuracy of identification. When the dominant eye is occluded, the occluder is preferably mechanically controlled or not occluded by the subject himself, so as to avoid the subject being mentally prepared when occluding the pupil, which affects the conditioned reflex effect.
[0180] The monocular action atlas refers to the image of the action changes of a single eye. For example, this embodiment can identify the action of the corresponding right eye when the left eye of a subject is blocked, and determine the dominant eye according to the change of the contralateral eye after the eyeball is blocked. When it is identified that the subject's right eye moves violently, it can be determined that the right eye movement is the action of the dominant eye being blocked, and the system determines that the blocked left eye is the dominant eye; when it is identified that the subject's right eye movement is slight, it can be determined that the right eye movement is the action of the auxiliary eye being blocked, and the system determines that the right eye is the dominant eye.
[0181] Among them, the recognition action method is obtained by including a convolutional neural network (CNN) or a fusion of multiple neural networks. The action refers to the distinguished actions when the dominant eye and the accessory eye are respectively occluded. It is not limited to the examples mentioned above, but can be continuously optimized through the training of deep neural networks to obtain more accurate actions. Further, by adopting the trained neural network recognition method, the dominant eye result can be obtained by inputting a monocular action atlas. It is only necessary to set the initial value and feed the corpus to obtain the input and output results, without considering the process, and the effect is significant.
[0182] As a preferred implementation in this embodiment, since the dominant eye plays an important role in depth perception and spatial positioning, its occlusion will affect the judgment of the distance and position of the object and the balance perception of the body. The scope of the action includes non-eye actions, including balance compensation actions. For example, during the test, the subject is required to perform a balance action, preferably standing on one foot. When the subject stands on one foot, the dominant eye is blocked compared to the non-dominant eye, and the impact on the subject's balance is different. Therefore, based on this preferred scheme, identifying the subject's balance compensation action can undoubtedly determine its dominant eye, wherein the balance compensation action refers to the subject's action to correct the body balance. For example, when the subject stands on one foot and the dominant eye is blocked, the subject will shake and swing the body more conditionedly to seek body rebalance. Based on the amplitude of the subject's balance compensation action, the subject's dominant eye discrimination result is determined. Therefore, compared with the prior art, this technology covers a scheme for determining the dominant eye by identifying the balance compensation action, which is non-obvious.
[0183] Based on this embodiment, the subjects do not need to determine the dominant eye in advance as in the prior art, and then the system recognizes and records it. This solution is easily affected by the subjective understanding of the subjects, and covering the wrong eye will delay the experimental results. In the scheme of this embodiment, no matter the subject covers the dominant eye or the accessory eye, the system can determine the correct dominant eye based on whether the covered eye is the left eye or the right eye, and whether the covered eye has the conditioned reflex action of the covered dominant eye. Therefore, this scheme eliminates the technical problem of the subjective factors of the subjects affecting the experiment, and the effect is significant.
[0184] S700: integrating the dominant eye discrimination result and the first identity correspondence to determine a second identity correspondence;
[0185] Specifically, the second identity relationship is the correspondence between the dominant eye discrimination result and the first identity, and the first identity relationship indicates the correspondence between the identity and the attribute of the monocular eye to be tested, that is, the correspondence between the occluded eye of the subject's identity as the left eye or the right eye can be obtained; the dominant eye discrimination result indicates the relationship between the dominant eye and the attribute of the monocular eye after the test, that is, the identity of the subject and the dominant eye discrimination result are associated with the monocular attribute, thereby obtaining a direct correspondence between the identity and the dominant eye. In a large-scale testing environment with a large number of subjects, this embodiment can directly obtain the dominant eye correspondence of all subjects by taking a single subject image, and can summarize and organize it.
[0186] S800: Summarizing the second identity correspondence relationship to generate a dominant eye and identity correspondence result;
[0187] Specifically, after obtaining the subject's identity information and dominant eye information, the system can obtain the second identity correspondence of all subjects in the subject's image, and after sorting out the second identity correspondence, generate the dominant eye and identity correspondence results of all subjects. Preferably, the dominant eye and identity correspondence results include but are not limited to establishing charts and establishing databases.
[0188] Embodiment 2:
[0189] This embodiment provides an automated dominant eye and dominant brain training method based on image analysis. Based on the same concept, this embodiment is used to implement an automated dominant eye and dominant brain determination method based on image analysis as described in Example 1, wherein the automated dominant eye and dominant brain training method based on image analysis is used to train the method for identifying the subject's actions described in step S600 of Example 1, including:
[0190] Obtain a single occluded image of the subject;
[0191] Based on the subject's occluded single image, obtaining the subject's identity information;
[0192] Based on the single image blocked by the subject, obtaining the dominant eye information of the subject;
[0193] Associating the dominant eye information with the subject's identity information to determine the dominant eye;
[0194] The determined dominant eye is used as corpus to train the automated dominant eye and dominant brain determination method based on image analysis, thereby reducing the loss of the dominant eye identity corresponding result relative to the determined dominant eye.
[0195] Specifically, in Embodiment 2, the subject occlusion single image is obtained by the following steps:
[0196] 1. Set up camera equipment in the test area;
[0197] 2. The subject enters the test area;
[0198] 3. The subject looks at the camera, places the cover between the eye and the camera, moves the cover until the cover overlaps with the camera, and remains still. If there are multiple subjects, wait until all subjects are still.
[0199] 4. Obtain a single occluded image of the subject, wherein the single occluded image of the subject is in the form of a photograph.
[0200] The principle of this step is that the brain will fuse the images of the two eyes into a three-dimensional image. The image of the dominant eye usually plays a dominant role in this process, while the image of the non-dominant eye plays an auxiliary role. When the occluder blocks both eyes separately, there will be differences in binocular vision, and the brain will take the dominant eye as the dominant and more reliable primary vision; for example, when the subject is looking at the camera lens and the occluder blocks the dominant eye, the visual imaging is that the occluder is aligned with the camera lens; when the occluder blocks the accessory eye, the visual imaging is that the occluder is invisible to the primary vision. According to the subject's visual perception, when the occluder coincides with the camera device, the occluded eye can be identified as the dominant eye.
[0201] Further, by identifying the subject's occluded single image, obtaining the subject's identity information and the subject's dominant eye information, the subject's dominant eye can be determined. Therefore, this embodiment can determine the dominant eye correspondence relationship of the subject without objection, and use the dominant eye information and the subject's identity information as corpus training examples with the described method of automatic dominant eye and dominant brain determination based on image analysis, compare the dominant eye result obtained by the method described in Example 1 with the dominant eye result obtained by the method described in Example 2, and determine to reduce the loss of the dominant eye identity correspondence result relative to the determined dominant eye. For example, according to this embodiment, it can be determined that the right eye of subject A is the dominant eye. When training this embodiment, the subject's conditioned reflex actions that occur when the dominant eye of the subject occludes the right eye, such as specific blinking actions, pupil focus changes, and gaze deviation actions, are captured as evidence for judging the dominant eye in the video test. The scheme described in Example 2 can also recognize images by the convolutional neural network (CNN) described in Example 1 or the fusion of CNN and multiple neural networks.
[0202] When testing the dominant eye through the existing technology, the tester and the subject are tested individually, which cannot be used for large-scale statistics and cannot be adapted to large sample scenarios. In addition, since the existing technology does not provide technical inspiration, any combination of the existing technology by those skilled in the art is intended to achieve one-on-one testing. Therefore, those skilled in the art have no motivation to think of the multiple implementations described and achieve equivalent technical effects.
[0203] According to the technical solution of this embodiment, those skilled in the art have been able to implement a method for testing the dominant eye of one or more subjects by taking a photo of the subject after covering one eye (called a static test). Therefore, the technical solution of this embodiment itself obviously has the ability to identify and test the dominant eye of multiple people. This embodiment is combined with the collected images of the subjects and used as a technical solution for training the dominant eye through video as described in Example 1 (called a dynamic test). Those skilled in the art can understand that the static test can be used as a separate solution for testing the dominant eye, and can also be used as a medium for training the dynamic test. In general, the better technical effects that the dynamic test can achieve are as follows:
[0204] 1. When testing to obtain the dominant eye data of multiple people, for example, when directly obtaining the dominant eye data of the entire class through a single shot, due to differences in the understanding ability and concentration of class members, a few people may occlude their dominant eyes by mistake. The dynamic test can identify the confidence of the subject's actions through an algorithm when the subject occludes the eye incorrectly. Regardless of whether the subject can follow the correct steps for the test, it can determine whether the subject occludes the eye incorrectly and make secondary corrections;
[0205] 2. When performing static testing, the subject may cover the wrong eye due to panic, discomfort, etc. when taking the countdown photo, which is not in a natural state, which may easily lead to poor error tolerance of the test data. The dynamic test method can make the subject more relaxed. The subject is in a relaxed and comfortable environment.
[0206] The subjects will have less psychological pressure, be more focused, and have faster nerve conduction speed, which will help to perform more natural conditioned reflexes and present more accurate test results.
[0207] 3. Dynamic testing: The algorithm can identify useful information in a large video. The useful information refers to the moment when the subject's single eye is blocked. In other words, when judging the dominant eye, only a video needs to be provided, and there may be a large amount of useless information in the video. The algorithm can automatically identify useful evidence for judging the dominant eye. Compared with static testing, the subject needs to provide test data accurately. The threshold for testing the dominant eye through video is lower, and more recognition scenarios can be identified.
[0208] Embodiment three:
[0209] An automated dominant eye and dominant brain determination system based on image analysis, used to implement an automated dominant eye and dominant brain determination method based on image analysis described in Embodiment 1, comprising:
[0210] A first acquisition module is used to acquire an image of a subject;
[0211] A first recognition module, used for recognizing key images based on the subject image;
[0212] A first obtaining module, configured to obtain a monocular attribute view based on the key image;
[0213] A second obtaining module, configured to obtain a facial attribute view based on the key image;
[0214] A first fusion module, configured to fuse the monocular attribute view and the facial attribute view to obtain a first identity correspondence;
[0215] A second recognition module is used to recognize the subject's actions based on the subject's image and obtain a dominant eye discrimination result;
[0216] A second fusion module is used to fuse the dominant eye discrimination result with the first identity correspondence to determine a second identity correspondence;
[0217] The first summarizing module is used to summarize the second identity correspondence relationship and generate a dominant eye and identity correspondence result.
[0218] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.
Claims
1. An automated dominant eye and dominant brain determination method based on image analysis, applied to an automated dominant eye and dominant brain determination system based on image analysis, characterized in that: The steps include: S100: Acquire the image of the subject; S200: identifying key images based on the subject image; S300: obtaining a monocular attribute view based on the key image; S400: Obtaining a facial attribute view based on the key image; S500: fusing the monocular attribute view and the facial attribute view to obtain a first identity correspondence relationship; S600: Based on the subject image, identify the subject's action and obtain a dominant eye discrimination result; S700: integrating the dominant eye discrimination result and the first identity correspondence to determine a second identity correspondence; S800: Summarizing the second identity correspondence relationship to generate a dominant eye and identity correspondence result; The subject images refer to a collection of multiple pictures covering the subject, and the pictures of the subject images are arranged in order.
2. The method for automatically determining the dominant eye and dominant brain based on image analysis as claimed in claim 1, characterized in that: The step S200 includes: S210: Identify the pupil position of the subject in the subject image; S220: traversing the subject's image, determining the change in the number of the subject's pupils based on the subject's pupil positions, and obtaining the key image; S230: Recording time information of the key picture; The key image refers to the image at the moment when the subject's pupil position is blocked.
3. The method for automatically determining the dominant eye and dominant brain based on image analysis as claimed in claim 2, characterized in that: The step S300 includes: S310: performing image recognition on the key image to obtain monocular attribute confidences, and sorting the monocular attribute confidences to obtain sorting results; S320: Obtaining monocular attributes based on the sorting result; S330: Based on the monocular attribute, modify the monocular attribute view; The single eye attribute includes the left eye or the right eye.
4. The method for automatically determining the dominant eye and dominant brain based on image analysis as claimed in claim 3, characterized in that: The step S310 includes: S311: Identifying monocular features based on the pupil position; S312: Obtaining a monocular feature confidence level according to the monocular feature; S313: weighting the monocular feature confidence to obtain the monocular attribute confidence; Among them, the monocular features include: tear gland position, eye canthus height, eye frame contour, and eye structure.
5. The method for automatically determining the dominant eye and dominant brain based on image analysis as claimed in claim 2, characterized in that: The step S400 includes: S410: Identify the key image, and establish a facial feature network by splitting facial features of the key image; S420: performing intelligent processing on the facial feature network to obtain facial attribute confidence; S430: Obtain a facial attribute view according to the facial attribute confidence.
6. The method for automatically determining the dominant eye and dominant brain based on image analysis as claimed in claim 5, characterized in that: The step S420 includes: S421: Identify the occluder, determine the possibility that the occluder is the target object according to the similarity between the occluder and the target object, and obtain the occluder identification result; S422: According to the result of the occlusion identification, the intelligent processing method is selected to obtain the facial attribute confidence; Wherein, the step S422 includes the following steps: S421: Importing and recording the subject's identity data, wherein the identity data includes a facial picture; S422: Identify the occluder, and determine the possibility that the occluder is the target object according to the similarity between the occluder and the target object, and obtain the occluder identification result; S423: Obtaining a threshold value, and selecting the optimal intelligent processing method according to the result of the occlusion identification to obtain the facial attribute confidence; Wherein, the step S423 includes the following steps: When the occluder identification result is that the occluder is a target object, the intelligent processing method is: identifying the facial feature network through a convolutional neural network and comparing it with the preprocessing result to obtain the facial attribute confidence, and ending step S423; When the occluder identification result is that the occluder is not the target object, determining the destruction rate of the facial feature being occluded; When the destruction rate is lower than the threshold, the intelligent processing method is: introducing an attention mechanism, ignoring the obscured facial features, identifying the facial feature network and weighting it to obtain a score, obtaining the facial attribute confidence according to the score, and ending step S423; When the destruction rate is not lower than the threshold, the intelligent processing method is: traversing the imported subject identity data, using the subject identity data as material, reconstructing the blocked part, and generating multiple virtual character portraits; identifying the similarity between the virtual character portrait and the subject identity data, and obtaining the facial attribute confidence; Among them, the occluder refers to a prop used to cover the pupil and conduct a dominant eye test; the target object refers to a designated occluder, and the characteristic information of the target object is stored in an automated dominant eye and dominant brain determination system based on image analysis; the preprocessing result refers to the preprocessing result obtained by superimposing the characteristics of the target object at the pupil position of the facial image in advance to form a preset occluded image.
7. The method for automatically determining the dominant eye and dominant brain based on image analysis as claimed in claim 6, characterized in that: The step S500 includes: S510: Retrieve key images; S520: Based on the key image, retrieve the pupil position of the subject; S530: Based on the pupil position of the subject, the monocular attribute view and the facial attribute view are fused to obtain a first identity correspondence.
8. The method for automatically determining the dominant eye and dominant brain based on image analysis as claimed in claim 7, characterized in that: The step S530 includes: S531: Based on the subject's pupil position, identifying close monocular positions, and obtaining a closeness ranking of the monocular positions; S532: Based on the monocular position proximity sorting, the monocular attribute views are merged, the wrong monocular position is eliminated, and the monocular position proximity sorting is corrected; S533: sorting based on the proximity of the monocular positions, fusing the facial attribute views, and determining the monocular position inclusion based on the facial feature network; S534: Based on the monocular position including the degree, associate the monocular attribute with the identity data to obtain a first identity correspondence.
9. The method for automatically determining the dominant eye and dominant brain based on image analysis as claimed in claim 8, characterized in that: The step S600 includes: S610: Taking the time information of the key image as a starting point, traversing the subject's image to generate a monocular action atlas; S620: Based on the monocular action atlas, recognize the action and determine the monocular attribute; S630: Obtaining a dominant eye discrimination result based on the monocular attribute; The action refers to the conditioned reflex action of the contralateral eyeball when one pupil is blocked.
10. An automated dominant eye and dominant brain training method based on image analysis, based on the same concept as the automated dominant eye and dominant brain determination method based on image analysis as claimed in any one of claims 1 to 9, characterized in that: The method for training the automated dominant eye and dominant brain based on image analysis is used to train the method for identifying the subject's movements in step S600, and includes: Obtain a single occluded image of the subject; Based on the subject's occluded single image, obtaining the subject's identity information; Based on the single image blocked by the subject, obtaining the dominant eye information of the subject; Associating the dominant eye information with the subject's identity information to determine the dominant eye; The determined dominant eye is used as corpus to train the automated dominant eye and dominant brain determination method based on image analysis, thereby reducing the loss of the dominant eye identity corresponding result relative to the determined dominant eye.
