Gift dispensing method and gift dispensing system based on robot vision

Abstract

The application provides a gift giving method based on robot vision, and relates to the technical field of robots, which comprises the following steps: shooting a monitoring video to obtain a monitoring video frame; determining a pre-target research personnel and a selection coefficient thereof according to the monitoring video frame; controlling a robot to meet the target research personnel; sending a research request and displaying a research questionnaire; shooting a face video in a response time period; determining a research quality index according to the face video; determining a gift selection index according to the research quality index and the selection coefficient; and determining a target gift according to the gift selection index. According to the application, the face video of the target research personnel can be shot in the process of response, and whether the target research personnel answers seriously can be analyzed, so that it is determined what kind of gift is given to the target research personnel. The facial expression of the target research personnel can be automatically monitored and the seriousness during the response can be automatically analyzed, the accuracy of video analysis and gift giving is improved, and the research quality is improved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a gift-giving method and system based on robot vision. Background Technology

[0002] During personnel surveys, the quality of responses to questionnaires varies. Some questionnaires reflect the respondents' true intentions, while others are merely perfunctory. Survey distributors can distribute gifts based on the respondents' responses. For example, if the distributor observes that the respondent is attentive, a higher-value gift can be given; conversely, if the respondent is perfunctory, a lower-value gift can be given. In related technologies, robots can assist survey distributors in conducting surveys. For instance, robots can provide questionnaires to personnel. However, robots struggle to determine whether respondents are attentive or whether their responses are valuable, thus making it difficult to decide what kind of gift to give. Summary of the Invention

[0003] This invention provides a gift distribution method based on robot vision, which can solve the problem that robots have difficulty in judging whether the respondent is serious during the answering process, whether the information in the answer is valuable, and also have difficulty in judging what kind of gift to give to the respondent.

[0004] According to a first aspect of the present invention, a gift distribution method based on robot vision is provided, comprising:

[0005] The surveillance camera captures video of a preset area and analyzes the video frames to obtain the surveillance video.

[0006] Based on the monitoring video frames, determine the target survey personnel suitable for conducting the survey within the preset area and the selection coefficient of the target survey personnel.

[0007] The robot was brought together with the target research personnel.

[0008] Send a survey request to the target survey personnel, and display the survey questionnaire on the display device after the target survey personnel accept the survey request;

[0009] During the time period from the start of the survey to the end of the survey, facial videos of the target survey personnel are taken.

[0010] Based on facial video, determine the survey quality indicators for the target survey personnel;

[0011] Based on the survey quality indicators and selection coefficients, determine the gift selection indicators;

[0012] Based on the gift selection criteria, identify the target gift, open the storage compartment containing the target gift, and remind the target research personnel to take the target gift.

[0013] According to the present invention, determining the target survey personnel suitable for conducting surveys within a preset area and the selection coefficient of the target survey personnel based on monitoring video frames includes:

[0014] The first video frame is obtained by sampling the surveillance video frames.

[0015] The first region where each person is located in the first video frame is determined by the image detection model;

[0016] The behavior recognition model is used to process the first region of each first video frame to obtain the behavior type recognition result of the person.

[0017] The first area is processed by a clothing recognition model to obtain the clothing type recognition result of the person;

[0018] Based on the behavior type recognition results and clothing type recognition results, the selection coefficient of the person is determined;

[0019] Based on the selection coefficients of each individual, target survey participants are selected from a pool of participants.

[0020] According to the present invention, determining the selection coefficient of a person based on the behavior type recognition result and the clothing type recognition result includes:

[0021] Based on the behavior type identification results, determine the first probability information that the person's current behavior belongs to multiple behavior types;

[0022] Based on the clothing type recognition result, a second probability information is determined to indicate that the clothing worn by the person belongs to a preset type of clothing.

[0023] Based on the first probability information and the second probability information, the selection coefficient of the personnel is determined.

[0024] According to the present invention, determining the selection coefficient of a person based on the first probability information and the second probability information includes:

[0025] According to the formula

[0026]

[0027] Determine the selection coefficient for the i-th person. ,in, The first probability information is given for the behavior being performed by the i-th person, which belongs to the j-th behavior type. The second probability information is given for the clothing worn by the i-th person, which belongs to a preset type of clothing. The selection weight is the weight corresponding to the j-th behavior type, where n is the number of behavior types, j≤n, and both j and n are positive integers.

[0028] According to the present invention, determining the survey quality indicators of target survey personnel based on facial video includes:

[0029] The facial video is analyzed to obtain multiple facial video frames;

[0030] Sample facial video frames to obtain the video frames to be processed;

[0031] By using the 2D convolutional neural network layer of the facial expression recognition model, each video frame to be processed is processed to obtain the first facial feature map of multiple feature channels of each video frame to be processed.

[0032] The first facial feature maps of multiple video frames to be processed with the same feature channel are combined to form a first facial feature map sequence corresponding to the feature channel;

[0033] The first facial feature map sequence corresponding to each feature channel is segmented according to time period to obtain multiple first facial feature map groups;

[0034] By using the 3D convolutional neural network hierarchy of the facial expression recognition model, each first facial feature map group is processed to obtain the first facial feature information of each first facial feature map group in each feature channel.

[0035] The first facial feature information of the first facial feature map group of multiple feature channels in the same time period is spliced ​​together to obtain the second facial feature information corresponding to the time period.

[0036] By splicing together the second facial feature information corresponding to each time period, facial expression recognition feature information is obtained.

[0037] Based on the first facial feature map, obtain the third facial feature information;

[0038] Arrange the third facial feature information in chronological order to obtain the third facial feature information sequence;

[0039] The third facial feature information sequence is processed by the 1D convolutional neural network layer of the facial expression recognition model to obtain dynamic facial expression feature information.

[0040] Facial expression recognition feature information and facial expression dynamic feature information are concatenated to obtain facial expression description information;

[0041] The facial expression description information is processed by the first multilayer perceptual network layer of the facial expression recognition model to obtain the facial expression focus recognition results of the target survey personnel.

[0042] Based on the facial expression focus recognition results, the survey quality indicators of the target researchers are determined.

[0043] According to the present invention, the training method of the facial expression recognition model includes:

[0044] Obtain sample video frames from the sample facial video;

[0045] By using the 2D convolutional neural network layers of the facial expression recognition model, the first training facial feature map of multiple feature channels of each sample video frame is obtained, and the sequence of the first training facial feature map of each feature channel is obtained.

[0046] Multiple sets of first training facial feature maps are obtained by processing the first training facial feature map sequence according to the time period.

[0047] The first training facial feature information of the first training facial feature map group is obtained through the 3D convolutional neural network layer of the facial expression recognition model.

[0048] The first training facial feature information of the same time period of multiple feature channels is spliced ​​together to obtain the second training facial feature information corresponding to the time period.

[0049] The second training facial feature information is processed by the second multi-layer perceptual network layer to obtain the third probability information of inattentive state in each time period.

[0050] The second training facial feature information corresponding to each time period is spliced ​​together to obtain the training facial expression recognition feature information.

[0051] Based on the first training facial feature map, obtain the third training facial feature information;

[0052] The first training facial feature information is arranged in chronological order to obtain the third training facial feature information sequence;

[0053] The third training facial feature information sequence is processed by the 1D convolutional neural network layer of the facial expression recognition model to obtain the dynamic feature information of the training facial expression.

[0054] The dynamic feature information of the training facial expressions is processed by the third multi-layer perceptual network layer to obtain the fourth probability information of maintaining focus on facial expressions.

[0055] The training facial expression recognition feature information and the training facial expression dynamic feature information are spliced ​​together to obtain the training facial expression description information.

[0056] The training facial expression description information is processed by the first multilayer perceptual network layer of the facial expression recognition model to obtain the training expression attention recognition result.

[0057] Based on the third probability information, the fourth probability information, the training results of facial expression attention recognition, and the annotation information of the sample facial videos, the loss function of the facial expression recognition model is obtained.

[0058] The facial expression recognition model is trained according to the loss function to obtain the trained facial expression recognition model.

[0059] According to the present invention, a loss function for a facial expression recognition model is obtained based on third probability information, fourth probability information, training expression focus recognition results, and annotation information of sample facial videos, including:

[0060] According to the formula

[0061]

[0062] Obtain the loss function LOSS of the facial expression recognition model, where, This represents the third probability information regarding the existence of a state of inattentiveness during the t-th time period. This represents the probability of an unfocused state occurring during the t-th time period, determined based on the annotation information of the sample facial videos. This is the fourth probability information. This provides probability information for maintaining focus based on facial expressions determined from the annotation information of sample facial videos. This refers to the probability information of facial expression focus during the survey process, determined based on the training results of facial expression focus recognition. This is the probability information of facial expression focus during the survey process, determined based on the annotation information of the sample facial videos. , , The preset weights are m, where m is the number of time periods, t≤m, and both t and m are positive integers.

[0063] According to the present invention, gift selection indicators are determined based on survey quality indicators and selection coefficients, including:

[0064] Determine the relationship between the survey quality indicators and the selection coefficient;

[0065] If the survey quality index is greater than or equal to the selection coefficient, then according to the formula...

[0066]

[0067] Define the gift selection criteria, where Q is the survey quality index, S is the selection coefficient, and C is the gift selection index.

[0068] If the survey quality index is less than the selection coefficient, then according to the formula...

[0069]

[0070] Determine the criteria for gift selection.

[0071] According to a second aspect of the present invention, a gift distribution system based on robot vision is provided, comprising:

[0072] The parsing module is used to capture surveillance video within a preset area using a surveillance camera and parse it to obtain surveillance video frames;

[0073] The selection coefficient module is used to determine the target survey personnel suitable for conducting surveys within a preset area and the selection coefficient of the target survey personnel based on the monitoring video frames.

[0074] The rendezvous module is used to control the robot to rendezvous with the target survey personnel.

[0075] The display module is used to send survey requests to the target survey personnel and display the survey questionnaire through the display device after the target survey personnel accept the survey request.

[0076] The camera module is used to capture facial videos of the target researchers during the response period from the start of their responses to the questionnaire to the end of their submission.

[0077] The survey quality indicators module is used to determine the survey quality indicators of the target survey personnel based on facial videos.

[0078] Gift selection indicators are used to determine gift selection criteria based on survey quality indicators and selection coefficients.

[0079] The Target Gift module is used to identify target gifts based on gift selection criteria, open the storage compartment containing the target gift, and remind the target research personnel to take the target gift.

[0080] By adopting the above technical solution, the present invention can achieve the following technical effects:

[0081] According to this invention, target survey participants can be screened using surveillance video captured by a camera capable of capturing images of a preset area. A robot is then controlled to display a survey questionnaire to the target participants. During the participants' responses, facial videos are captured and analyzed to determine whether they are answering attentively, thereby deciding what kind of gift to give them. The system automatically monitors and analyzes the participants' facial expressions and the level of attentiveness during the responses, improving the accuracy of video analysis and gift distribution, thus enhancing survey quality. When determining the selection coefficient, the system analyzes the participants' behavior and clothing to determine whether their actions are conducive to participation in the survey and whether their clothing indicates they are staff members. This comprehensive assessment of both factors improves the accuracy and objectivity of the selection coefficient. When determining survey quality indicators, a 2D convolutional neural network layer can be used to obtain the first facial feature map of each video frame to be processed, and a 3D convolutional neural network layer can be used to obtain the second facial feature information corresponding to the time period. This is used to determine whether the expression of the target survey participant is serious or attentive in each time period. A 1D convolutional neural network layer can also be used to obtain dynamic facial expression feature information to determine the changes in the target survey participant's expression throughout the time period. Combining these two aspects of information allows for a comprehensive description of the target survey participant's expression focus, improving the accuracy of expression focus recognition results and survey quality indicators. During the training of the facial expression recognition model, the model's ability to judge the existence of inattentiveness in each time period, its ability to recognize expression changes, and its ability to recognize expression focus are trained separately. This comprehensively improves the model's accuracy and enhances the accuracy of the facial expression recognition model in judging whether the target survey participant is answering seriously. When determining the target gift, the expected survey quality and the actual survey quality are compared based on the survey quality indicators and selection coefficients. This determines whether the target survey participant's answer exceeds expectations, thus determining a gift that matches the quality of their answer. This helps to increase the enthusiasm of survey participants and improves the quality of subsequent surveys. Attached Figure Description

[0082] Figure 1 A schematic flowchart of a gift distribution method based on robot vision according to an embodiment of the present invention is shown as an example.

[0083] Figure 2 An exemplary schematic diagram illustrating the determination of survey quality indicators for target researchers according to an embodiment of the present invention is shown.

[0084] Figure 3 A block diagram of a robot vision-based gift distribution system according to an embodiment of the present invention is shown as an example. Detailed Implementation

[0085] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0086] Figure 1 An exemplary flowchart illustrates a gift-giving method based on robot vision according to an embodiment of the present invention, the method comprising:

[0087] Step S1: Capture surveillance video within a preset area using a surveillance camera and analyze the video to obtain surveillance video frames;

[0088] Step S2: Based on the monitoring video frames, determine the target survey personnel suitable for conducting the survey within the preset area and the selection coefficient of the target survey personnel.

[0089] Step S3: Control the robot to meet with the target research personnel;

[0090] Step S4: Send a survey request to the target survey personnel, and after the target survey personnel accept the survey request, display the survey questionnaire through the display device;

[0091] Step S5: During the time period from the start of the target researcher's response to the questionnaire to the end of the questionnaire submission, a video of the target researcher's face is captured.

[0092] Step S6: Determine the survey quality indicators for the target survey personnel based on the facial video.

[0093] Step S7: Determine the gift selection indicators based on the survey quality indicators and selection coefficients;

[0094] Step S8: Based on the gift selection criteria, determine the target gift, open the storage compartment containing the target gift, and remind the target survey personnel to take the target gift.

[0095] According to an embodiment of the present invention, a gift distribution method based on robot vision can screen target survey participants by capturing surveillance video from a camera capable of capturing images of a preset area. The method then controls a robot to display a survey questionnaire to the target participants. During the participants' responses, facial video is captured and analyzed to determine whether they are answering attentively, thereby deciding what gift to distribute. This method automatically monitors the participants' facial expressions and analyzes their level of attentiveness during responses, improving the accuracy of video analysis and gift distribution, and contributing to higher survey quality.

[0096] According to one embodiment of the present invention, in step S1, a monitoring camera is set in a preset area such as a meeting room, capable of capturing monitoring video within the preset area. This monitoring camera is a high-definition camera, capable of clearly capturing the clothing of personnel. When the robot has completed its survey of the previous target researcher, or when the robot is in a task-free state, it can acquire monitoring video from the current moment and a previous period (e.g., 5-10 seconds), and analyze this monitoring video to obtain multiple monitoring video frames. Based on these multiple monitoring video frames within a time period, the behavior of each person can be analyzed to select suitable target researchers for the survey.

[0097] According to one embodiment of the present invention, in step S2, multiple people may exist within a preset area, and their behavior can be analyzed through multiple monitoring video frames to screen target survey personnel. For example, if a person is currently not doing anything and may be waiting for someone, or if a person is taking a aimless walk, then this person is suitable as a target survey personnel; conversely, if a person is busy with other things, then this person is not suitable as a target survey personnel.

[0098] According to one embodiment of the present invention, determining target personnel suitable for investigation within a preset area and selection coefficients for the target personnel based on surveillance video frames includes: sampling the surveillance video frames to obtain a first video frame; determining a first region where each person is located in the first video frame using an image detection model; processing the first region of each person in each first video frame using a behavior recognition model to obtain behavior type recognition results for the person; processing the first region using a clothing recognition model to obtain clothing type recognition results for the person; determining selection coefficients for the person based on the behavior type recognition results and clothing type recognition results; and selecting target personnel from multiple personnel based on the selection coefficients for each person.

[0099] According to one embodiment of the present invention, in order to reduce the amount of computation and improve computational efficiency, multiple monitoring video frames can be sampled, for example, once every 0.2 seconds, so that 5 first video frames can be obtained per second. The image detection model is a deep learning neural network model, such as the YOLO model, which can determine the first region where each person is located in the first video frame, and can also determine the people present in all first video frames. The first regions of these people in multiple first video frames can be analyzed to determine what kind of behavior these people are performing.

[0100] According to one embodiment of the present invention, the behavior recognition model is a deep learning neural network model, such as the DensePose model, which can be used to process the first region of a person in each first video frame to obtain the behavior type recognition result of the person, such as the probability information of the person's behavior belonging to multiple behavior types.

[0101] According to one embodiment of the present invention, in addition to recognizing personnel behavior, personnel clothing can also be identified. For example, a meeting staff member may be standing still, not engaged in any activity, but should not be identified as a target research participant. The clothing of each person can be identified; if the type of clothing matches a preset type for meeting staff, that person should not be identified as a target research participant. The clothing recognition model is a convolutional neural network model, which can determine the clothing type recognition result, that is, the probability information of determining whether a person's clothing matches a preset type of meeting staff clothing.

[0102] According to one embodiment of the present invention, a selection coefficient for each person can be determined based on their behavior and attire, and the selection coefficient can be used to determine whether the person can be selected as a target survey participant.

[0103] According to one embodiment of the present invention, determining a person's selection coefficient based on the behavior type identification result and the clothing type identification result includes: determining first probability information that the person's current behavior belongs to multiple behavior types based on the behavior type identification result; determining second probability information that the clothing worn by the person belongs to a preset type of clothing based on the clothing type identification result; and determining the person's selection coefficient based on the first probability information and the second probability information.

[0104] According to an embodiment of the present invention, as described above, the behavior type identification result is the probability information that a person's behavior belongs to multiple behavior types, namely, the first probability information; the clothing type identification result is the probability information that a person's clothing belongs to a preset type of clothing for venue staff, namely, the second probability information.

[0105] According to an embodiment of the present invention, determining the selection coefficient of a person based on the first probability information and the second probability information includes: determining the selection coefficient of the i-th person according to formula (1). ,

[0106] (1)

[0107] in, The first probability information is given for the behavior being performed by the i-th person, which belongs to the j-th behavior type. The second probability information is given for the clothing worn by the i-th person, which belongs to a preset type of clothing. The selection weight is the weight corresponding to the j-th behavior type, where n is the number of behavior types, j≤n, and both j and n are positive integers.

[0108] According to one embodiment of the present invention, the ease of participation in a survey varies depending on the individual's current activity. For example, if a person is taking a walk and not busy with other things, then that person is suitable to participate in the survey; conversely, if a person is reading, then that person is not suitable to participate in the survey. Therefore, different weights can be assigned to different activities, i.e., selection weights, to indicate the ease of participation in the survey when a person is engaging in that activity. For example, a selection weight of 0.8 for walking indicates that a person engaging in walking is relatively suitable to participate in the survey, while a selection weight of 0.3 for reading indicates that a person engaging in reading is not suitable to participate in the survey. The ease coefficient of participation in the survey can be obtained by weighting and summing the probability of a person engaging in various activities with the corresponding selection weights. This indicates the degree of suitability of a person to participate in the survey, determined by their ongoing behavior.

[0109] According to one embodiment of the present invention, This information represents the probability that the person is not wearing the preset type of clothing, and can also represent the probability that the person is not a staff member at the venue. As mentioned above, if the person is a staff member, then that person is not suitable as a target for the survey; conversely, if not, then that person can be a target for the survey. Therefore, This indicates the degree of suitability of a person to participate in the survey, as determined by their attire.

[0110] According to one embodiment of the present invention, the suitability of a person to participate in the survey, determined by the person's current behavior, is multiplied by the suitability determined by the person's clothing to obtain a selection coefficient. Furthermore, the person corresponding to the maximum selection coefficient can be identified as the target survey participant, and the selection coefficient of the target survey participant can be recorded.

[0111] In this way, we can analyze people's behavior and clothing to determine whether they are suitable for participating in the survey based on their current behavior and whether they are staff members based on their clothing. This comprehensive assessment of both factors improves the accuracy and objectivity of the selection criteria.

[0112] According to an embodiment of the present invention, in step S3, after selecting the target survey personnel, the robot's moving component can be controlled to move the robot to the vicinity of the target survey personnel. For example, the coordinate position of the target survey personnel in the preset area can be determined based on the first area of ​​the target survey personnel in the last monitoring video frame and the calibration parameters of the monitoring camera. The coordinate position of the robot in the preset area can be determined in the same way, and then the robot can be controlled to move from its current coordinate position to the vicinity of the target survey personnel's coordinate position, for example, a position 0.3 meters away from the target survey personnel's coordinate position and directly facing the target survey personnel's face.

[0113] According to one embodiment of the present invention, in step S4, a survey request can be sent to the target survey participant. For example, the survey request can be displayed on a display device, showing the text message "May I invite you to participate in the survey?" and displaying accept and reject buttons. If the target survey participant presses the accept button, it means that the target survey participant accepts the survey, and the survey questionnaire can be displayed on the display device. Otherwise, if the target survey participant presses the reject button, it means that the target survey participant rejects the survey, and the above steps can be repeated to find other participants as the target survey participants.

[0114] According to one embodiment of the present invention, in step S5, during the time period when the target researcher answers the survey questionnaire, the robot's camera can capture facial video of the target researcher, and the facial video can be analyzed to determine whether the target researcher is answering seriously.

[0115] Figure 2 An exemplary schematic diagram illustrating the determination of survey quality indicators for target researchers according to an embodiment of the present invention is shown.

[0116] According to an embodiment of the present invention, in step S6, determining the survey quality indicators of the target survey personnel based on the facial video includes: parsing the facial video to obtain multiple facial video frames; sampling the facial video frames to obtain video frames to be processed; processing each video frame to be processed through a 2D convolutional neural network layer of a facial expression recognition model to obtain first facial feature maps of multiple feature channels for each video frame to be processed; combining the first facial feature maps of multiple video frames to be processed with the same feature channel into a first facial feature map sequence corresponding to the feature channel; segmenting the first facial feature map sequence corresponding to each feature channel according to a time period to obtain multiple first facial feature map groups; processing each first facial feature map group through a 3D convolutional neural network layer of a facial expression recognition model to obtain first facial feature information of each first facial feature map group of each feature channel; and combining the multiple feature channels... The first facial feature information of the first facial feature map group of the same time period is spliced ​​to obtain the second facial feature information corresponding to the time period; the second facial feature information corresponding to each time period is spliced ​​to obtain facial expression recognition feature information; the third facial feature information is obtained based on the first facial feature map; the third facial feature information is arranged in chronological order to obtain the third facial feature information sequence; the third facial feature information sequence is processed through the 1D convolutional neural network layer of the facial expression recognition model to obtain facial expression dynamic feature information; the facial expression recognition feature information and the facial expression dynamic feature information are spliced ​​to obtain facial expression description information; the facial expression description information is processed through the first multilayer perceptual network layer of the facial expression recognition model to obtain the expression focus recognition result of the target researcher; the research quality index of the target researcher is determined based on the expression focus recognition result.

[0117] According to one embodiment of the present invention, facial videos can be parsed to obtain multiple facial video frames. However, the response time for each person is different, therefore, the length of the facial video and the number of facial video frames are not fixed values. Therefore, facial video frames can be sampled to obtain a specific number of video frames to be processed. For example, if the facial video is 3 minutes long, 5 video frames can be sampled per second, resulting in 900 video frames to be processed. Or, for example, if the facial video is 5 minutes long, 3 video frames can be sampled per second, also resulting in 900 video frames to be processed. By standardizing the number of video frames to be processed, the input requirements of subsequent neural network models can be met.

[0118] According to one embodiment of the present invention, a 2D convolutional neural network layer can perform 2D convolution processing on all images to be processed to obtain first facial feature maps of multiple feature channels of the images to be processed, and can perform the same processing on multiple images to be processed to obtain first facial feature maps of multiple feature channels of each video frame to be processed.

[0119] According to one embodiment of the present invention, in the same feature channel, there are first facial feature maps of multiple video frames to be processed. These first facial feature maps can be arranged in the time order of their corresponding video frames to be processed to obtain the first facial feature map sequence of the feature channel. Similarly, each feature channel corresponds to a first facial feature map sequence.

[0120] According to one embodiment of the present invention, the sequence of first facial feature maps corresponding to a feature channel is segmented according to time periods to obtain multiple groups of first facial feature maps. This processing can be performed on the sequence of first facial feature maps corresponding to each feature channel to obtain multiple groups of first facial feature maps corresponding to each feature channel. The time periods corresponding to the same order of first facial feature map groups in different feature channels are the same. That is, first facial feature map groups with different feature channels but the same order each include multiple first facial feature maps. The first facial feature maps with the same sequence number in each group correspond to the same video frame to be processed, and also to the shooting time of the same video frame to be processed. Therefore, the shooting times corresponding to the first facial feature maps with the same sequence number in different feature channels are the same. Thus, the time periods corresponding to the same order of first facial feature map groups in different feature channels are the same. Therefore, the first facial feature map groups with the same order in multiple feature channels represent multiple feature descriptions of the facial expressions of the target researcher during the same time period. In the example, the sequence of first facial feature maps corresponding to each feature channel can be divided into 100 groups of first facial feature maps. The present invention does not limit the specific number of first facial feature map groups.

[0121] According to one embodiment of the present invention, each first facial feature map group corresponding to each feature channel can be processed by a 3D convolutional neural network layer. The 3D convolutional neural network layer may include 3D convolutional layers and pooling layers, which can process each first facial feature map group to obtain the first facial feature information of each first facial feature map group. The first facial feature information is feature information in vector form.

[0122] According to one embodiment of the present invention, first facial feature information corresponding to the same time period in different feature channels can be spliced ​​together, for example, in a side-by-side manner, to obtain second facial feature information. The second facial feature information is matrix-based; for example, each first facial feature information is a 1×64 vector, and the number of feature channels is 64. The first facial feature information corresponding to the same time period in each feature channel can be spliced ​​together in a side-by-side manner, that is, each first facial feature information is used as a column vector in the matrix, resulting in a 64×64 matrix, i.e., the second facial feature information. The second facial feature information for a time period contains multiple feature representations of the target researcher's expression during that time period. By performing the above processing on the first facial feature information of multiple feature channels for each time period, the second facial feature information for each time period can be obtained. This can be used to indicate whether the expression during each time period is serious or not, or whether the person is disturbed or giving a perfunctory answer, resulting in an unserious or unfocused expression.

[0123] According to one embodiment of the present invention, the second facial feature information corresponding to each time period can be spliced ​​together to obtain facial expression recognition feature information, which can indicate whether the expression of the target researcher is serious or solemn within each time period. For example, each second facial feature information is a 64×64 matrix, and the number of time periods is 100. After splicing, a 6400×64 matrix can be obtained, that is, facial expression recognition feature information.

[0124] According to one embodiment of the present invention, a first facial feature map of multiple feature channels corresponding to the same video frame to be processed can be processed by another 2D convolutional neural network layer. This 2D convolutional neural network layer includes 2D convolutional layers and pooling layers, and can obtain third facial feature information corresponding to the video frame to be processed. Furthermore, the same processing can be performed on the first facial feature maps of multiple feature channels corresponding to each video frame to be processed to obtain third facial feature information corresponding to each video frame to be processed. The third facial feature information is in vector form. The third facial feature information can be arranged according to the chronological order of the video frames to be processed to obtain a sequence of third facial feature information.

[0125] According to one embodiment of the present invention, in order to dynamically acquire the facial expression features of a target researcher and understand the changes in their facial expressions, a 1D convolutional neural network layer can be used to process the third facial feature information sequence to obtain dynamic facial expression feature information in vector form. In the example, the 1D convolutional neural network layer may include 1D convolutional layers and fully connected layers, which can obtain dynamic facial expression feature information to describe the dynamic changes in the facial expressions of the target researcher between the shooting times of different video frames to be processed.

[0126] According to one embodiment of the present invention, facial expression recognition feature information and facial expression dynamic feature information can be concatenated to obtain facial expression description information. For example, the facial expression recognition feature information is a 6400×64 matrix, and the facial expression dynamic feature information is a 6400×1 vector. After concatenation, a 6400×65 matrix can be obtained, which is the facial expression description information. The facial expression description information can include the expression state over multiple time periods (i.e., whether it is serious or not focused) and the expression changes throughout the entire time period, which can be used to comprehensively describe the focus level of the target researcher's expression.

[0127] According to one embodiment of the present invention, the first multilayer perceptron layer may include fully connected layers and activation layers. The activation layers may process the input values ​​using a sigmoid activation function. For example, multiple fully connected layers may process the facial expression description information to obtain expression focus recognition values. The activation layers may then process these expression focus recognition values ​​to obtain the expression focus recognition results of the target researcher. This information can be used to represent the probability that the target researcher's expression is focused, or the probability that the target researcher's answers to the questions in the questionnaire are valuable. The expression focus recognition results can be used as a survey quality indicator for the target researcher.

[0128] In this way, a 2D convolutional neural network layer can be used to obtain the first facial feature map of each video frame to be processed, and a 3D convolutional neural network layer can be used to obtain the second facial feature information corresponding to the time period. This is used to determine whether the expression of the target researcher is serious or attentive in each time period. Furthermore, a 1D convolutional neural network layer can be used to obtain dynamic facial expression feature information to determine the changes in the expression of the target researcher throughout the entire time period. Thus, the two aspects of information can be combined to comprehensively describe the expression focus of the target researcher, thereby improving the accuracy of expression focus recognition results and survey quality indicators.

[0129] According to an embodiment of the present invention, the above-mentioned facial expression recognition model can be trained before use. The training method of the facial expression recognition model includes: acquiring sample video frames of sample facial videos; obtaining first training facial feature maps of multiple feature channels for each sample video frame through a 2D convolutional neural network layer of the facial expression recognition model, and obtaining a sequence of first training facial feature maps for each feature channel; obtaining multiple first training facial feature map groups according to time periods for the first training facial feature map sequences; obtaining first training facial feature information of the first training facial feature map groups through a 3D convolutional neural network layer of the facial expression recognition model; concatenating the first training facial feature information of the same time period for multiple feature channels to obtain second training facial feature information corresponding to the time period; processing the second training facial feature information through a second multilayer perceptron layer to obtain third probability information of the existence of an inattentive state within each time period; and concatenating the second training facial feature information corresponding to each time period to obtain training facial expression recognition feature information. Based on the first training facial feature map, first training facial feature information is obtained; the first training facial feature information is arranged in chronological order to obtain a first training facial feature information sequence; the first training facial feature information sequence is processed through a 1D convolutional neural network layer of the facial expression recognition model to obtain training facial expression dynamic feature information; the training facial expression dynamic feature information is processed through a third multilayer perceptron layer to obtain fourth probability information on facial expression focus; the training facial expression recognition feature information and the training facial expression dynamic feature information are concatenated to obtain training facial expression description information; the training facial expression description information is processed through a first multilayer perceptron layer of the facial expression recognition model to obtain training expression focus recognition result; based on the third probability information, the fourth probability information, the training expression focus recognition result, and the annotation information of the sample facial video, the loss function of the facial expression recognition model is obtained; the facial expression recognition model is trained according to the loss function to obtain the trained facial expression recognition model.

[0130] According to one embodiment of the present invention, sample facial videos of trainees can be captured. During the capture of the sample facial videos, the trainees can maintain a serious expression or exhibit changes in expression. Furthermore, sample video frames can be acquired in a manner similar to that used for the video frames to be processed, which will not be elaborated further here.

[0131] According to one embodiment of the present invention, during the training process, the acquisition methods of the first training facial feature map sequence and the first training facial feature map group are similar to those of the aforementioned first facial feature map sequence and the first facial feature map group. The acquisition methods of the first training facial feature information and the second training facial feature information are similar to those of the aforementioned first facial feature information and second facial feature information, and will not be repeated here. Similar to the second facial feature information, the second training facial feature information can represent whether the expression in each time period is serious or not, or whether the expression is distracted or perfunctory, resulting in an unfocused or inattentive expression. The second training facial feature information can be processed by a second multilayer perceptron layer to obtain a third probability information indicating the presence of an unfocused state in each time period. For example, the second multilayer perceptron layer may include a fully connected layer and an activation layer. The fully connected layer can map the second training facial feature information to the input value of the activation layer. The activation layer can use the sigmoid activation function to process the input value to obtain probability information, i.e., the third probability information, which can be used to represent the probability of an unfocused state in each time period.

[0132] According to one embodiment of the present invention, the training facial expression recognition feature information, the third training facial feature information, the third training facial feature information sequence, and the training facial expression dynamic feature information are obtained in a similar manner to the aforementioned facial expression recognition feature information, the third facial feature information, the third facial feature information sequence, and the facial expression dynamic feature information, respectively, and will not be described in detail here. Similar to the facial expression dynamic feature information, the facial expression dynamic feature information can also represent the dynamic changes in the facial expressions of the target researcher between the shooting times of different video frames to be processed. The training facial expression dynamic feature information can be processed by a third multilayer perceptron layer to obtain a fourth probability information indicating that the facial expression remains focused. For example, the third multilayer perceptron layer may include a fully connected layer and an activation layer. The fully connected layer can map the training facial expression dynamic feature information to the input value of the activation layer. The activation layer can use the sigmoid activation function to process the input value to obtain probability information, i.e., the fourth probability information, which can be used to represent the probability of maintaining focus when facial expression changes are small.

[0133] According to one embodiment of the present invention, the training facial expression description information is obtained in a similar manner to the acquisition of the above facial expression description information. The training facial expression description information can be input into the first multilayer perceptual network layer to obtain the training expression focus recognition result, which can be used to represent the probability information of the trainee's expression maintaining focus.

[0134] According to one embodiment of the present invention, the loss function of the facial expression recognition model is obtained based on the third probability information, the fourth probability information, the training expression focus recognition result, and the annotation information of the sample facial video, including: obtaining the loss function LOSS of the facial expression recognition model according to formula (2).

[0135] (2)

[0136] in, This represents the third probability information regarding the existence of a state of inattentiveness during the t-th time period. This represents the probability of an unfocused state occurring during the t-th time period, determined based on the annotation information of the sample facial videos. This is the fourth probability information. This provides probability information for maintaining focus based on facial expressions determined from the annotation information of sample facial videos. This refers to the probability information of facial expression focus during the survey process, determined based on the training results of facial expression focus recognition. This is the probability information of facial expression focus during the survey process, determined based on the annotation information of the sample facial videos. , , The preset weights are m, where m is the number of time periods, t≤m, and both t and m are positive integers.

[0137] According to one embodiment of the present invention, , and All information is manually labeled. Under the condition of inattentiveness during the t-th time period, ,otherwise, , for and The cross-entropy loss function between them This represents the cross-entropy loss function across multiple time periods. During training, this cross-entropy loss function can be reduced across multiple time periods, thereby... and This improves the model's accuracy in judging whether there is a lack of focus in different time periods, enhances the accuracy of the second training facial feature information, and improves the model's precision.

[0138] According to one embodiment of the present invention, when facial expression changes are minimal across multiple sample video frames and the user remains focused for most of the time, ,otherwise, . for and The cross-entropy loss function can be reduced during training to make it... and This improves the model's accuracy in recognizing facial expression changes, enhances the accuracy of training facial expression dynamic feature information, and improves model precision.

[0139] According to one embodiment of the present invention, when the facial expression is serious and focused during the research process, ,otherwise, . for and The cross-entropy loss function can be reduced during training to make it... and This closer proximity improves the model's accuracy in recognizing facial expressions and focus, thus enhancing the model's precision.

[0140] According to one embodiment of the present invention, the loss function of the facial expression recognition model is obtained by weighted summation of the above three cross-entropy loss functions. The loss function can be used for backpropagation, and the parameters of the facial expression recognition model can be adjusted by gradient descent to train the facial expression recognition model and improve its accuracy. The above training steps can be executed multiple times until the accuracy of the facial expression recognition model meets the requirements. For example, if the accuracy of the training expression attention recognition result output by the facial expression recognition model exceeds a preset threshold when tested on a test set, the training can be completed and the trained facial expression recognition model can be obtained.

[0141] In this way, during the training process of the facial expression recognition model, the model's ability to judge whether there is a lack of focus in different time periods, the model's ability to recognize changes in facial expressions, and the model's ability to recognize the level of focus in facial expressions can be trained separately. This comprehensively improves the model's accuracy and enhances the accuracy of the facial expression recognition model in judging whether the target researcher is answering seriously.

[0142] According to an embodiment of the present invention, in step S7, determining the gift selection index based on the survey quality index and the selection coefficient includes: determining the magnitude relationship between the survey quality index and the selection coefficient; if the survey quality index is greater than or equal to the selection coefficient, then determining the gift selection index according to formula (3).

[0143] (3)

[0144] Where Q is the survey quality index, S is the selection coefficient, and C is the gift selection index; if the survey quality index is less than the selection coefficient, then the gift selection index is determined according to formula (4):

[0145] (4).

[0146] According to one embodiment of the present invention, the selection coefficient can be used as a basis for selecting target survey participants, and also as an expected quality indicator of the target survey participants' responses to the survey questionnaire. For example, selecting a person who is convenient to participate in the survey as a target survey participant is based on the expectation that this person is not busy with other matters, is convenient to participate in the survey, and can provide high-quality responses. Therefore, the selection coefficient can be used as an expected quality indicator of the target survey participants' responses to the survey questionnaire. The survey quality indicator is the quality indicator actually monitored during the process of the target survey participants answering the survey questionnaire. It can reflect the seriousness of the target survey participants when answering the questions, and can also reflect the actual quality of their answers. Therefore, if the survey quality indicator is greater than or equal to the selection coefficient, it means that the actually monitored quality indicator exceeds expectations, and can be passed... By amplifying the survey quality indicators, a higher gift selection index C is obtained. Conversely, if the survey quality index is lower than the selection coefficient, it indicates that the actual monitored quality indicators are below expectations, which can be addressed through... By narrowing down the quality indicators of the survey, we obtained a lower value for the gift selection indicator C.

[0147] According to one embodiment of the present invention, in step S8, a target gift can be determined based on a gift selection index. For example, three value ranges can be set: less than 0.8, greater than or equal to 0.8 and less than 1.2, and greater than 1.2. Gifts corresponding to each value range are set accordingly. For example, the value range less than 0.8 is set with the lowest-value gift, the value range of 0.8 and equal to 0.8 and less than 1.2 is set with a medium-value gift, and the value range greater than 1.2 is set with the highest-value gift. This determines which value range the gift selection index belongs to, thus determining the target gift. For example, if the gift selection index is 1.25, which falls within the value range greater than 1.2, then the target gift is the aforementioned highest-value gift. After determining the target gift, the storage compartment containing the target gift can be opened, and the target survey personnel can be reminded to take the target gift.

[0148] In this way, when determining target gifts, the expected and actual survey quality can be compared based on survey quality indicators and selection coefficients. This helps determine whether the responses of the target survey participants exceed expectations, thereby identifying gifts that match the quality of their responses. This approach helps to increase the enthusiasm of survey participants and improve the quality of subsequent surveys.

[0149] According to an embodiment of the present invention, a gift distribution method based on robot vision can screen target survey participants by capturing surveillance video from a camera capable of capturing images of a preset area. The method then controls a robot to display a survey questionnaire to the target participants. During the participants' responses, facial video is captured and analyzed to determine whether they are answering attentively, thereby determining which gift to distribute. The method automatically monitors and analyzes the participants' facial expressions and their level of attentiveness during responses, improving the accuracy of video analysis and gift distribution, and contributing to higher survey quality. When determining the selection coefficient, the method analyzes the participants' behavior and clothing to determine whether their actions are conducive to participation in the survey and whether their clothing indicates they are event staff. This comprehensive assessment of both factors enhances the accuracy and objectivity of the selection coefficient. When determining survey quality indicators, a 2D convolutional neural network layer can be used to obtain the first facial feature map of each video frame to be processed, and a 3D convolutional neural network layer can be used to obtain the second facial feature information corresponding to the time period. This is used to determine whether the expression of the target survey participant is serious or attentive in each time period. A 1D convolutional neural network layer can also be used to obtain dynamic facial expression feature information to determine the changes in the target survey participant's expression throughout the time period. Combining these two aspects of information allows for a comprehensive description of the target survey participant's expression focus, improving the accuracy of expression focus recognition results and survey quality indicators. During the training of the facial expression recognition model, the model's ability to judge the existence of inattentiveness in each time period, its ability to recognize expression changes, and its ability to recognize expression focus are trained separately. This comprehensively improves the model's accuracy and enhances the accuracy of the facial expression recognition model in judging whether the target survey participant is answering seriously. When determining the target gift, the expected survey quality and the actual survey quality are compared based on the survey quality indicators and selection coefficients. This determines whether the target survey participant's answer exceeds expectations, thus determining a gift that matches the quality of their answer. This helps to increase the enthusiasm of survey participants and improves the quality of subsequent surveys.

[0150] Figure 3 An exemplary block diagram of a robot vision-based gift distribution system according to an embodiment of the present invention is shown, the system comprising:

[0151] The parsing module is used to capture surveillance video within a preset area using a surveillance camera and parse it to obtain surveillance video frames;

[0152] The selection coefficient module is used to determine the target survey personnel suitable for conducting surveys within a preset area and the selection coefficient of the target survey personnel based on the monitoring video frames.

[0153] The rendezvous module is used to control the robot to rendezvous with the target survey personnel.

[0154] The display module is used to send survey requests to the target survey personnel and display the survey questionnaire through the display device after the target survey personnel accept the survey request.

[0155] The camera module is used to capture facial videos of the target researchers during the response period from the start of their responses to the questionnaire to the end of their submission.

[0156] The survey quality indicators module is used to determine the survey quality indicators of the target survey personnel based on facial videos.

[0157] Gift selection indicators are used to determine gift selection criteria based on survey quality indicators and selection coefficients.

[0158] The Target Gift module is used to identify target gifts based on gift selection criteria, open the storage compartment containing the target gift, and remind the target research personnel to take the target gift.

[0159] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0160] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and any variations or modifications may be made to the implementation of the present invention without departing from the stated principles.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gift distribution method based on robot vision, characterized in that, include: The surveillance camera captures video of a preset area and analyzes the video frames to obtain the surveillance video. Based on the monitoring video frames, determine the target survey personnel suitable for conducting the survey within the preset area and the selection coefficient of the target survey personnel. The robot was brought together with the target research personnel. Send a survey request to the target survey personnel, and display the survey questionnaire on the display device after the target survey personnel accept the survey request; During the time period from the start of the survey to the end of the survey, facial videos of the target survey personnel are taken. Based on facial video, determine the survey quality indicators for the target survey personnel; Based on the survey quality indicators and selection coefficients, determine the gift selection indicators; Based on the gift selection criteria, identify the target gift, open the storage compartment containing the target gift, and remind the target research personnel to take the target gift.

2. The gift distribution method based on robot vision according to claim 1, characterized in that, Based on the surveillance video frames, determine the target survey personnel suitable for conducting surveys within the preset area, as well as the selection coefficients for these target survey personnel, including: The first video frame is obtained by sampling the surveillance video frames. The first region where each person is located in the first video frame is determined by the image detection model; The behavior recognition model is used to process the first region of each first video frame to obtain the behavior type recognition result of the person. The first area is processed by a clothing recognition model to obtain the clothing type recognition result of the person; Based on the behavior type recognition results and clothing type recognition results, the selection coefficient of the person is determined; Based on the selection coefficients of each individual, target survey participants are selected from a pool of participants.

3. The gift distribution method based on robot vision according to claim 2, characterized in that, Based on the behavior type recognition results and clothing type recognition results, the selection coefficient for each person is determined, including: Based on the behavior type identification results, determine the first probability information that the person's current behavior belongs to multiple behavior types; Based on the clothing type recognition result, a second probability information is determined to indicate that the clothing worn by the person belongs to a preset type of clothing. Based on the first probability information and the second probability information, the selection coefficient of the personnel is determined.

4. The gift distribution method based on robot vision according to claim 3, characterized in that, Based on the first probability information and the second probability information, the selection coefficient for the personnel is determined, including: According to the formula Determine the selection coefficient for the i-th person. ,in, The first probability information is given for the behavior being performed by the i-th person, which belongs to the j-th behavior type. The second probability information is given for the clothing worn by the i-th person, which belongs to a preset type of clothing. The selection weight is the weight corresponding to the j-th behavior type, where n is the number of behavior types, j≤n, and both j and n are positive integers.

5. The gift distribution method based on robot vision according to claim 1, characterized in that, Based on facial video, determine the survey quality indicators for the target researchers, including: The facial video is analyzed to obtain multiple facial video frames; Sample facial video frames to obtain the video frames to be processed; By using the 2D convolutional neural network layer of the facial expression recognition model, each video frame to be processed is processed to obtain the first facial feature map of multiple feature channels of each video frame to be processed. The first facial feature maps of multiple video frames to be processed with the same feature channel are combined to form a first facial feature map sequence corresponding to the feature channel; The first facial feature map sequence corresponding to each feature channel is segmented according to time period to obtain multiple first facial feature map groups; By using the 3D convolutional neural network hierarchy of the facial expression recognition model, each first facial feature map group is processed to obtain the first facial feature information of each first facial feature map group in each feature channel. The first facial feature information of the first facial feature map group of multiple feature channels in the same time period is spliced ​​together to obtain the second facial feature information corresponding to the time period. By splicing together the second facial feature information corresponding to each time period, facial expression recognition feature information is obtained. Based on the first facial feature map, obtain the third facial feature information; Arrange the third facial feature information in chronological order to obtain the third facial feature information sequence; The third facial feature information sequence is processed by the 1D convolutional neural network layer of the facial expression recognition model to obtain dynamic facial expression feature information. Facial expression recognition feature information and facial expression dynamic feature information are concatenated to obtain facial expression description information; The facial expression description information is processed by the first multilayer perceptual network layer of the facial expression recognition model to obtain the facial expression focus recognition results of the target survey personnel. Based on the facial expression focus recognition results, the survey quality indicators of the target researchers are determined.

6. The gift distribution method based on robot vision according to claim 5, characterized in that, The training method for the facial expression recognition model includes: Obtain sample video frames from the sample facial video; By using the 2D convolutional neural network layers of the facial expression recognition model, the first training facial feature map of multiple feature channels of each sample video frame is obtained, and the sequence of the first training facial feature map of each feature channel is obtained. Multiple sets of first training facial feature maps are obtained by processing the first training facial feature map sequence according to the time period. The first training facial feature information of the first training facial feature map group is obtained through the 3D convolutional neural network layer of the facial expression recognition model. The first training facial feature information of the same time period of multiple feature channels is spliced ​​together to obtain the second training facial feature information corresponding to the time period. The second training facial feature information is processed by the second multi-layer perceptual network layer to obtain the third probability information of inattentive state in each time period. The second training facial feature information corresponding to each time period is spliced ​​together to obtain the training facial expression recognition feature information. Based on the first training facial feature map, obtain the third training facial feature information; The first training facial feature information is arranged in chronological order to obtain the third training facial feature information sequence; The third training facial feature information sequence is processed by the 1D convolutional neural network layer of the facial expression recognition model to obtain the dynamic feature information of the training facial expression. The dynamic feature information of the training facial expressions is processed by the third multi-layer perceptual network layer to obtain the fourth probability information of maintaining focus on facial expressions. The training facial expression recognition feature information and the training facial expression dynamic feature information are spliced ​​together to obtain the training facial expression description information. The training facial expression description information is processed by the first multilayer perceptual network layer of the facial expression recognition model to obtain the training expression attention recognition result. Based on the third probability information, the fourth probability information, the training results of facial expression attention recognition, and the annotation information of the sample facial videos, the loss function of the facial expression recognition model is obtained. The facial expression recognition model is trained according to the loss function to obtain the trained facial expression recognition model.

7. The gift distribution method based on robot vision according to claim 6, characterized in that, Based on the third probability information, the fourth probability information, the training results of facial expression attention recognition, and the annotation information of the sample facial videos, the loss function of the facial expression recognition model is obtained, including: According to the formula Obtain the loss function LOSS of the facial expression recognition model, where, This represents the third probability information regarding the existence of a state of inattentiveness during the t-th time period. This represents the probability of an unfocused state occurring during the t-th time period, determined based on the annotation information of the sample facial videos. This is the fourth probability information. This provides probability information for maintaining focus based on facial expressions determined from the annotation information of sample facial videos. This refers to the probability information of facial expression focus during the survey process, determined based on the training results of facial expression focus recognition. This is the probability information of facial expression focus during the survey process, determined based on the annotation information of the sample facial videos. , , The preset weights are m, where m is the number of time periods, t≤m, and both t and m are positive integers.

8. The gift distribution method based on robot vision according to claim 1, characterized in that, Based on the survey quality indicators and selection coefficients, the gift selection indicators were determined, including: Determine the relationship between the survey quality indicators and the selection coefficient; If the survey quality index is greater than or equal to the selection coefficient, then according to the formula... Define the gift selection criteria, where Q is the survey quality index, S is the selection coefficient, and C is the gift selection index. If the survey quality index is less than the selection coefficient, then according to the formula... Determine the criteria for gift selection.

9. A gift distribution system based on robot vision, characterized in that, include: The parsing module is used to capture surveillance video within a preset area using a surveillance camera and parse it to obtain surveillance video frames; The selection coefficient module is used to determine the target survey personnel suitable for conducting surveys within a preset area and the selection coefficient of the target survey personnel based on the monitoring video frames. The rendezvous module is used to control the robot to rendezvous with the target survey personnel. The display module is used to send survey requests to the target survey personnel and display the survey questionnaire through the display device after the target survey personnel accept the survey request. The camera module is used to capture facial videos of the target researchers during the response period from the start of their responses to the questionnaire to the end of their submission. The survey quality indicators module is used to determine the survey quality indicators of the target survey personnel based on facial videos. Gift selection indicators are used to determine gift selection criteria based on survey quality indicators and selection coefficients. The Target Gift module is used to identify target gifts based on gift selection criteria, open the storage compartment containing the target gift, and remind the target research personnel to take the target gift.

Images