Automatic conference investigation robot

Abstract

The invention provides an automatic conference investigation robot, which relates to the technical field of robots and comprises a display device, a moving component, a camera, a communication component and a processor, the processor is used for acquiring a monitoring image and screening an undetermined investigation object; according to the monitoring image, determining first position information of the to-be-investigated objects and second position information of the conference automatic investigation robot, and selecting an investigation object from the to-be-investigated objects; moving to a preset position around the investigation object, and displaying an investigation questionnaire; a face image of the investigation object is taken, and a reference value score of the answer information is determined. According to the invention, the monitoring image of the conference site can be acquired through the communication component, so that a suitable investigation object is selected, and the enthusiasm and the seriousness of the investigation object participating in the investigation can be analyzed through the facial image, so that the reference value of the answer information filled by the investigation object is determined, and the accuracy and objectivity of the investigation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robots, and in particular to an automatic conference research robot. Background Art

[0002] In the related art, CN117464641A discloses a robot capable of remote conferencing and patrolling, including a chassis, a mobile component arranged below the chassis, a power supply component arranged in the chassis, an outer cover arranged above the chassis, a conference system host arranged in the chassis and located in the outer cover, a laser radar arranged above the outer cover, a lifting module arranged above the outer cover, a telescopic module arranged above the lifting module, a patrol camera arranged below the telescopic module, and a conference camera arranged on the telescopic module, wherein the lifting module and the laser radar are fixedly installed above the outer cover, and the conference camera and the patrol camera are fixedly installed on the telescopic module. The beneficial effect of this scheme is that the scheme allows workers to remotely understand the on-site situation more clearly, can find problems in time, and can find the cause faster according to the recent situation recorded by the patrol camera. It can hold a real-time meeting with the on-site remotely, can communicate more efficiently, and improve work efficiency.

[0003] CN109015638B discloses a patrol robot control method and system for conference security, the method comprising: controlling a first patrol robot to reach each patrol point on the patrol route according to a preset patrol route, and obtaining the flow of people at each patrol point; dividing the patrol priorities of each patrol point according to the flow of people at each patrol point; planning the patrol path and the patrol duration of each patrol point according to the patrol priority of each patrol point; controlling a second patrol robot to reach each patrol point on the patrol path according to the patrol path, and performing abnormal event detection at each patrol point corresponding to the patrol duration of each patrol point, and sending an alarm notification to a preset terminal when an abnormal event is detected at a certain patrol point.

[0004] CN108724220B discloses a robot used for conferences, including a bottom plate, a protective pad and a robot body, the top of the bottom plate is fixedly connected with an annular plate, the top of the annular plate is fixedly connected with a protective frame, the outer surface of the annular plate is fixedly connected with a buffer spring, and the end of the buffer spring away from the annular plate is fixedly connected with a circular plate, the inner surface of the annular plate is fixedly connected with a spring rod, and one end of the spring rod is fixedly connected with a mounting plate, the side of the mounting plate away from the spring rod is fixedly connected to the outer surface of the protective pad, and both sides of the mounting plate are movably connected with elastic plates, and the scheme relates to the field of robot technology. The robot used for conferences can achieve a certain protection effect on the robot body, prevent the robot body from being easily damaged after a collision, reduce unnecessary economic costs, and create good conditions for subsequent normal use.

[0005] Therefore, in the related technology, the robot at the conference site can take pictures of the conference site, and can also move and patrol the conference site, but the robot in the related technology cannot interact and conduct research with the people at the conference site, and cannot automatically select research subjects.

[0006] The information disclosed in the background technology section of this application is only intended to deepen the understanding of the general background technology of this application, and should not be regarded as an admission or any form of suggestion that the information constitutes the prior art already known to those skilled in the art. Summary of the invention

[0007] The present invention provides an automatic conference research robot, which can solve the technical problems that the related technology cannot automatically select research objects and cannot interact and conduct research with people at the conference site.

[0008] According to a first aspect of the present invention, there is provided a conference automatic research robot, comprising: Display devices, mobile components, cameras, communication components and processors; The display device is used to display the survey questionnaire according to the instructions of the processor and receive the answer information to the survey questionnaire; The moving component is used to move to a preset position around the research object according to the instruction of the processor; The camera is used to capture scene images in the conference room and send the scene images to the processor, and is also used to capture portrait images of the research subjects and send the portrait images to the processor; The communication component is used to communicate with the surveillance camera in the conference venue and obtain the surveillance images taken by the surveillance camera, and is also used to communicate with the backend server; The processor is used to: Upon receiving a survey instruction from the backend server, the monitoring images at multiple times are acquired through the communication component; Screening a candidate for investigation from among multiple objects in the conference venue according to the surveillance image; Determine, based on the monitoring image, first position information of the subject to be investigated in the conference venue and second position information of the conference automatic investigation robot in the conference venue, wherein the first position information and the second position information are spatial position information in the conference venue; selecting a research subject from among the pending research subjects according to the first location information and the second location information; According to the first position information, the second position information and the scene image captured by the camera in real time, controlling the mobile component to move to a preset position around the survey object, and displaying the survey questionnaire through the display device; At multiple moments when the research subject inputs answer information, controlling the camera to capture the research subject's facial image; Determining a reference value score of the answer information based on the facial image; The answer information and the reference value score are sent to the backend server through the communication component.

[0009] By adopting the above technical solution, the present invention can achieve the following technical effects: According to the present invention, the monitoring images of the conference site can be obtained through the communication component, so as to select suitable research subjects, and the system can move near the research subjects, interact with the research subjects through the display device, guide the research subjects to participate in the research, and improve the convenience of the research activities. Based on the facial images of the research subjects taken by the camera, the enthusiasm and seriousness of the research subjects in participating in the research can be analyzed, so as to determine the reference value of the answer information filled in by the research subjects, and improve the accuracy and objectivity of the research. When screening the pending research subjects, the facial orientation information of the subject and the distance from other subjects can be used to jointly determine whether the subject has interactive activities with other subjects and whether it is suitable as a research subject, thereby improving the accuracy of screening the pending research subjects. When screening survey subjects, the intersection of the pending straight path and the area occupied by the subject can be used to determine whether there are obstacles on the pending straight path where the conference automatic survey robot moves to the first position information where the pending survey subject is located, and the difficulty coefficient of the conference automatic survey robot's movement is determined based on the number of obstacles. The movement difficulty score of the conference automatic survey robot to the first position information where each pending survey subject is located is comprehensively determined based on the difficulty coefficient and the length of the pending straight path, thereby improving the accuracy, objectivity and comprehensiveness of the movement difficulty score, and the pending survey subject with the lowest movement difficulty score can be selected as the survey subject, thereby improving the convenience of the survey and reducing the movement difficulty of the conference automatic survey robot. When determining the reference value score, the reference value score of the answer information input by the survey subject can be determined based on the expression category information of the survey subject when entering the answer information of each question, the time spent on entering the answer information of each question, and the score of each question, thereby comprehensively and objectively describing the reference value of the answer information.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present invention. Other features and aspects of the present invention will become more apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0012] Figure 1 A schematic diagram of an automatic conference research robot according to an embodiment of the present invention is exemplarily shown. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0015] Figure 1 A schematic diagram of an automatic conference research robot according to an embodiment of the present invention is exemplarily shown, wherein the automatic conference research robot comprises: a display device, a mobile component, a camera, a communication component and a processor; The display device is used to display the survey questionnaire according to the instructions of the processor and receive the answer information to the survey questionnaire; The moving component is used to move to a preset position around the research object according to the instruction of the processor; The camera is used to capture scene images in the conference room and send the scene images to the processor, and is also used to capture portrait images of the research subjects and send the portrait images to the processor; The communication component is used to communicate with the surveillance camera in the conference venue and obtain the surveillance images taken by the surveillance camera, and is also used to communicate with the backend server; The processor is used to: Upon receiving a survey instruction from the backend server, the monitoring images at multiple times are acquired through the communication component; Screening a candidate for investigation from among multiple objects in the conference venue according to the surveillance image; Determine, based on the monitoring image, first position information of the subject to be investigated in the conference venue and second position information of the conference automatic investigation robot in the conference venue, wherein the first position information and the second position information are spatial position information in the conference venue; selecting a research subject from among the pending research subjects according to the first location information and the second location information; According to the first position information, the second position information and the scene image captured by the camera in real time, controlling the mobile component to move to a preset position around the survey object, and displaying the survey questionnaire through the display device; At multiple moments when the research subject inputs answer information, controlling the camera to capture the research subject's facial image; Determining a reference value score of the answer information based on the facial image; The answer information and the reference value score are sent to the backend server through the communication component.

[0016] According to the embodiment of the present invention, the automatic conference survey robot can obtain monitoring images of the conference site through the communication component to select a suitable survey subject, and can move to the vicinity of the survey subject, interact with the survey subject through the display device, guide the survey subject to participate in the survey, and improve the convenience of the survey activity. It can also analyze the survey subject's enthusiasm and seriousness in participating in the survey based on the facial image of the survey subject taken by the camera, so as to determine the reference value of the answer information filled in by the survey subject and improve the accuracy and objectivity of the survey.

[0017] According to one embodiment of the present invention, the display device may be a touch screen, which may be used to display information, for example, display a survey questionnaire, and may also be used to input information, for example, receive touch input from a survey subject to obtain answer information.

[0018] According to one embodiment of the present invention, the moving component may include a driving motor, a roller component, a direction component, etc. The driving motor can drive the roller component to rotate so that the conference automatic survey robot moves forward. The direction component can be used to adjust the forward direction of the conference automatic survey robot. For example, the direction angle of the roller component can be adjusted to adjust the orientation of the roller component to adjust the forward direction of the conference automatic survey robot.

[0019] According to one embodiment of the present invention, a camera may be disposed above a display device so that when the subject of a survey enters answer information, the camera can be directed toward the subject's face to capture the subject's facial image. The camera may also capture scene images so that when the conference automatic survey robot moves forward, obstacles on the forward route may be captured. After the processor identifies the obstacle, the mobile component may be controlled to change direction to avoid the obstacle.

[0020] According to one embodiment of the present invention, the communication component may include a Bluetooth communication component, a WIFI communication component, etc. The present invention does not limit the type of the communication component. The communication component can be used to communicate with the surveillance camera in the conference venue. The surveillance image captured by the surveillance camera can be sent to the processor of the conference automatic survey robot through the communication component. In addition, the communication component can communicate with the background server to receive instructions from the background server, or send information obtained from the survey to the background server.

[0021] According to one embodiment of the present invention, the conference automatic survey robot may also include a battery assembly, a shell assembly, etc. The battery assembly can be used to provide power to the above-mentioned display device, mobile assembly, camera, communication assembly and processor, and the shell assembly is used to wrap and protect the battery assembly, processor and communication assembly.

[0022] According to one embodiment of the present invention, the background server may send a survey instruction to the conference automatic survey robot. In addition, the background server may send a survey questionnaire to the conference automatic survey robot, or the conference automatic survey robot may store the survey questionnaire in its own memory. When the processor receives the survey instruction, it may obtain monitoring images at multiple times through the communication component. For example, when the processor receives the survey instruction, it may obtain multiple frames of monitoring images within 5 seconds from the moment, thereby determining which object is suitable for participating in the survey based on the multiple frames of monitoring images within 5 seconds.

[0023] According to one embodiment of the present invention, based on the surveillance image, the candidate research subjects are screened from multiple objects in a conference venue, including: detecting the positions of the objects in the surveillance image through a trained image detection model to obtain a selection box for selecting each object, wherein the selection box is a minimum rectangular box for selecting the object; processing the image blocks selected by each selection box through a trained orientation recognition model to determine the facial orientation information of each object, wherein the facial orientation information is a direction vector with the centroid of the facial area of ​​the object as the starting point; and screening the candidate research subjects from multiple objects in the conference venue according to the centroid coordinates of the selection box and the facial orientation information.

[0024] According to one embodiment of the present invention, the image detection model can be a deep learning neural network model, which can be used to detect the location of objects in the surveillance image, select each object using a selection box, and mark each object so that each object has a unique identifier, thereby facilitating the distinction between different objects.

[0025] According to one embodiment of the present invention, the orientation recognition model can be a deep learning neural network model, which can determine the facial orientation information of each object, that is, in the monitoring image, the direction in which the face of each object is facing can be represented by a direction vector. In the example, the orientation recognition model can detect the facial key points of each object, and determine the facial orientation based on the positional relationship between the facial key points. For example, the key points of the eyes, nose and mouth can be detected, and the first connecting line of the two eye key points can be determined, and the midpoint of the first connecting line can be found, and the midpoint of the first connecting line can be connected with the mouth key point to obtain a second connecting line. Furthermore, a perpendicular line can be made through the nose key point to the second connecting line, and a unit vector parallel to the perpendicular line and pointing away from the face is the direction vector, which can be used as the facial orientation information of the object.

[0026] According to an embodiment of the present invention, based on the centroid coordinates of the selection box and the facial orientation information, the subject to be determined for investigation is selected from multiple subjects in the conference venue, including: obtaining a first screening condition C1 according to formula (1), (1) in, is the facial orientation information of the jth object in the ith surveillance image, is the centroid coordinate of the selection box of the jth object in the i-th monitoring image, is the centroid coordinate of the selection box of the kth object in the ith monitoring image, for The transposed vector of is the preset angle, and is the logical "and" operator, For the preset distance, is a preset number, if is a conditional function, m is the number of surveillance images at multiple moments, n is the number of objects in the surveillance images, i≤m, j≤n, k≤n, j≠k, and i, j, k, m and n are all positive integers; if the j-th object meets the first screening condition C1, the j-th object is determined as the pending survey object.

[0027] According to one embodiment of the present invention, in formula (1), the conditional function Indicated in and If , the conditional function value is 1, otherwise it is 0. is the cosine similarity between the facial orientation information of the jth object and the vector between the locations of the kth object and the jth object, is the angle between the facial orientation information of the j-th object and the vector between the positions of the k-th object and the j-th object. If the angle is less than or equal to the preset angle, it means that the j-th object faces the k-th object. Indicates the distance between the jth object and the kth object. If the distance is less than or equal to the preset distance, it means that the jth object is close to the kth object and there may be an association. If both conditions are met at the same time, that is, the jth object faces the kth object and the jth object is associated with the kth object, it means that there is an interactive activity between the jth object and the kth object. For example, the jth object and the kth object are talking. In other words, the jth object is communicating with the kth object, and it is not convenient for the jth object to enter the answer information as a survey subject. In this case, the conditional function value is 1, otherwise, the conditional function value is 0.

[0028] According to one embodiment of the present invention, It indicates the number of surveillance images in which the j-th object and the k-th object are detected to have interactive activities among multiple surveillance images. It represents the total number of surveillance images in which the jth object has interactive activities with all other objects. If the total number is less than or equal to the preset number, it means that the jth object has less communication with other objects, spends more time alone, and does not participate in other activities. The jth object meets the first screening condition C1. Therefore, the jth object can be determined as a pending research object.

[0029] In this way, the facial orientation information and the distance from other objects of the object can be used to jointly determine whether the object has interactive activities with other objects and whether it is suitable as a research subject, thereby improving the accuracy of screening pending research subjects.

[0030] According to one embodiment of the present invention, after the pending research objects are screened out, the first position information of the pending research objects in the conference venue and the second position information of the conference automatic research robot in the conference venue can be determined based on the last frame of the monitoring image among the multiple monitoring images acquired. For example, the coordinates of each pending research object can be determined in the picture of the monitoring image, and the first position information of the coordinates of the pending research object in the conference venue can be determined by the pre-calibrated internal and external parameters of the monitoring camera. Similarly, the second position information of the conference automatic research robot in the conference venue can be determined. The first position information and the second position information are spatial position information, that is, three-dimensional position information. In order to simplify the calculation, the Z-axis coordinate in the vertical direction can also be omitted to obtain the two-dimensional plane position information in the conference venue.

[0031] According to one embodiment of the present invention, there may be multiple pending survey objects, from which one survey object can be screened out for subsequent survey processing. According to the first position information and the second position information, a survey object is selected from the pending survey objects, including: according to the first position information and the second position information, determining the pending straight line path between the conference automatic survey robot and the first position information of each pending survey object; according to the monitoring image, determining the third position information of each object in the conference venue; according to the pending straight line path and the third position information, determining the movement difficulty score of the conference automatic survey robot to the first position information of each pending survey object; determining the pending survey object corresponding to the minimum value of the movement difficulty score as the survey object.

[0032] According to one embodiment of the present invention, the processor can construct a virtual two-dimensional coordinate system to represent the two-dimensional coordinate system of the plane of the conference venue. The coordinates of each object and the conference automatic survey robot in the two-dimensional coordinate system of the plane of the conference venue are the same as the coordinates in the virtual coordinate system, so that the processor can perform calculations. Based on the above two-dimensional first position information and second position information after removing the Z-axis coordinate, the undetermined straight line path can be determined, that is, the straight line segment obtained by connecting the first position information and the second position information in the virtual coordinate system. Furthermore, the coordinates of each object can be determined in the last frame of the monitoring image, and the third position information of each object in the conference venue can be obtained through the internal and external parameters of the monitoring camera, and the same processing is used to remove the Z-axis coordinate to determine its coordinates in the virtual coordinate system.

[0033] According to one embodiment of the present invention, the difficulty score of the conference automatic survey robot moving to the first position information where each pending survey object is located is determined based on the pending straight line path and the third position information, including: determining the occupied area of ​​each object in the conference venue based on the third position information; determining the difficulty score of the conference automatic survey robot moving to the first position information where the sth pending survey object is located based on formula (2) , (2) in, is the undetermined straight line path between the conference automatic research robot and the first position information of the sth undetermined research object, for Length, is the area occupied by the tth object, is an empty set, if is a conditional function, n is the number of objects in the monitoring image, is the preset scale factor, , t≤n, and t and n are both positive integers.

[0034] According to an embodiment of the present invention, the area occupied by each object in the conference venue may be a cylindrical area. In the virtual two-dimensional coordinate system, the area occupied is a circular area with the third position information as the center and a radius of a preset value.

[0035] According to one embodiment of the present invention, in formula (2), the conditional function It means that if the intersection of the area occupied by the t-th object and the undetermined straight line path between the conference automatic research robot and the first position information of the s-th undetermined research object is not an empty set, that is, the undetermined straight line path passes through the area occupied by the t-th object, then the conditional function value is 1, otherwise it is 0. Therefore, It can represent the number of occupied areas crossed by the pending straight path, or the number of obstacles (i.e., other objects) that prevent the conference automatic survey robot from moving to the location of the sth pending survey object. Every time an obstacle is passed, the automatic survey robot needs to adjust its direction and bypass it. Therefore, every time an obstacle is passed, the difficulty of the conference automatic survey robot moving to the location of the sth pending survey object will increase. The more obstacles there are, the greater the difficulty of moving. Therefore, As the difficulty coefficient for the conference automatic survey robot to move to the location of the sth pending survey object, the difficulty coefficient is greater than or equal to 1. When there are no obstacles on the pending straight path, the difficulty coefficient is equal to 1. If the obstacles increase, the difficulty coefficient will increase accordingly. On the other hand, the length of the pending straight path is also a factor for the conference automatic survey robot to move to the location of the sth pending survey object. The longer the length, the longer the time required for movement, the more power consumed for movement, and the greater the possibility that the survey object will change position over a longer period of time. Therefore, the longer the length of the pending straight path, the greater the difficulty of movement. The above difficulty coefficient can be multiplied by the length of the pending straight path to obtain a moving difficulty score, thereby comprehensively and accurately describing the difficulty of the conference automatic survey robot moving to the location of the sth pending survey object.

[0036] According to one embodiment of the present invention, the above method can be used to determine the movement difficulty score of the automatic survey robot to the location of each pending survey object, and the pending survey object corresponding to the minimum value of the movement difficulty score is determined as the survey object, that is, the difficulty of the conference automatic survey robot moving to the location of the survey object is the lowest.

[0037] In this way, it is possible to determine whether there are obstacles on the pending straight path of the conference automatic survey robot when it moves to the first position information of the pending survey object through the intersection of the pending straight path and the area occupied by the object, and determine the difficulty coefficient of the movement of the conference automatic survey robot based on the number of obstacles, and comprehensively determine the movement difficulty score of the conference automatic survey robot to the first position information of each pending survey object based on the difficulty coefficient and the length of the pending straight path, thereby improving the accuracy, objectivity and comprehensiveness of the movement difficulty score, and selecting the pending survey object with the lowest movement difficulty score as the survey object, thereby improving the convenience of the survey and reducing the movement difficulty of the conference automatic survey robot.

[0038] According to one embodiment of the present invention, after determining the survey subject, the moving path of the conference automatic survey robot can be determined based on the second position information of the conference automatic survey robot and the first position information of the survey subject, as well as obstacles on the to-be-determined straight path, so as to control the moving component to move according to the moving path, and during the movement, the scene image captured in real time by the camera is used to determine whether there are other obstacles in front, and if there are other obstacles, they are avoided, and finally the conference automatic survey robot can be moved to a preset position around the survey subject, for example, the conference automatic survey robot is moved to a preset distance in front of the survey subject, and the display device is facing the survey subject's face, thereby improving the convenience for the survey subject to view the survey questionnaire and enter answer information.

[0039] According to one embodiment of the present invention, at multiple moments when the survey subject inputs answer information, the camera can be controlled to capture the facial image of the survey subject. In addition, the multiple facial images captured during the time period when the survey subject inputs answer information can be grouped. For example, the moment when the survey questionnaire starts to be displayed is taken as the starting moment when the survey subject inputs answer information. At the X1th moment thereafter, the survey subject completes answering the first question, and the time period between the starting moment and the X1th moment is taken as the time period when the survey subject inputs the answer information of the first question in the survey questionnaire. Similarly, at the X2th moment thereafter, the survey subject completes answering the second question, and the time period between the X1th moment and the X2th moment is taken as the time period when the survey subject inputs the answer information of the second question in the survey questionnaire... Based on the above division method, the time period when the survey subject inputs the answer information can be divided into the time period when the survey subject inputs the answer information of multiple questions in the survey questionnaire, and the multiple facial images can be grouped according to the time period, that is, divided into the facial image groups obtained during the time period when the survey subject inputs the answer information of each question in the survey questionnaire.

[0040] According to one embodiment of the present invention, determining the reference value score of the answer information based on the facial image includes: determining the expression category information of the facial images at multiple moments through a trained expression detection model; and determining the reference value score of the answer information based on the expression category information.

[0041] According to one embodiment of the present invention, the reference value score can be used to describe whether the survey subject actively and seriously answers the questions in the survey questionnaire. The higher the enthusiasm of the survey subject and the more seriously he answers the questions, the greater the reference value of the survey subject's answer information to the survey questions. Conversely, if the survey subject does not answer seriously but perfunctorily, the authenticity of his answer information is lower and the reference value is also lower.

[0042] According to one embodiment of the present invention, an expression detection model can be used to detect expression category information of a facial image. For example, expression categories may include a serious expression, a bored expression, etc. The expression category information may be information in the form of a vector, which can be used to describe the type of expression in the facial image.

[0043] According to an embodiment of the present invention, determining the reference value score of the answer information according to the facial image and the expression category information includes: determining the reference value score V of the answer information according to formula (3), (3) in, The expression category information of the facial image at the tth moment within the time period when the research subject inputs the answer information of the hth question in the research questionnaire, For preset expression category information, is the score of the hth question in the survey questionnaire, is the total score of the survey questionnaire, The number of moments in the time period during which the respondents input their answers to the hth question in the survey questionnaire, is the preset time number of the hth question, H is the number of questions in the survey questionnaire, t≤ , h≤H, and t, h, and H are both positive integers.

[0044] According to one embodiment of the present invention, in formula (3), is the cosine similarity between the expression category information of the facial image and the preset expression category information. The preset expression category information may be the expression category information of a serious expression. Therefore, this item can be used to indicate the similarity between the expression category information of the facial image and the expression category information of a serious expression, and can also be used to indicate the seriousness of the subject in the facial image. It can represent the average seriousness of the respondents during the period of time when they input the answer information for the hth question. It is the ratio between the number of times during the time period when the survey subject inputs the answer information for the h-th question and the preset number of times for the h-th question. The preset number of times can be set for each question according to the difficulty level. A larger preset number of times can be set for questions with higher difficulty levels, and a smaller preset number of times can be set for questions with lower difficulty levels. If the ratio is small, it means that the time spent by the survey subject to answer the h-th question is much less than the preset number of times for the question, and the survey subject is more likely to give a perfunctory answer. Conversely, the larger the ratio is, the more serious the survey subject is. is the proportion of the score of the hth question in the total score. The higher the proportion, the more important the question is, the more significant it is for the research activity, and the greater the impact on the reference value of the answer information of the research questionnaire. Therefore, the above three items can be multiplied to get , which can be used as the reference value of the answer information of the hth question. The reference values ​​of the answer information of each question are summed up to get the reference value score of the answer information. The higher the reference value score, the more seriously the survey subject answers, and the greater the reference value of the answer information. For example, the greater the reference value for the subsequent improvement of the venue experience.

[0045] In this way, the reference value score of the answer information entered by the respondents can be determined based on the expression category information of the respondents when entering the answer information for each question, the time spent entering the answer information for each question, and the score of each question. This can comprehensively and objectively describe the reference value of the answer information.

[0046] According to one embodiment of the present invention, after obtaining the answer information and the reference value score, the processor can send it to the background server through the communication component. The background server can filter the answer information with higher reference value from the answer information of multiple survey subjects according to the reference value score, as a reference opinion for subsequent improvements, for example, a reference opinion for improving the environment and services at the conference site.

[0047] According to one embodiment of the present invention, the conference automatic research robot further includes: a multi-level storage component, the multi-level storage component includes multiple storage spaces, each for storing multiple types of gifts. For example, the shell of the conference automatic research robot includes multiple storage boxes, each storage box stores different types of gifts, and the gifts in the higher level storage boxes are more valuable.

[0048] According to one embodiment of the present invention, the processor is also used to: after the process of the survey subject inputting the answer information is completed, select the target storage space corresponding to the reference value score from the multiple storage spaces of the multi-level storage component, open the target storage space, and display a prompt message to take away the gift on the display device. In the example, each storage space corresponds to an interval of the reference value score, and the interval to which the actually calculated reference value score belongs can be determined, so that the target storage space corresponding to the interval can be opened, for example, the storage box corresponding to the interval is opened. Furthermore, a prompt message to take away the gift can also be displayed on the display device, or the conference automatic survey robot can also include an audio device, and a prompt message can be played through the audio device to prompt the survey subject to take away the gift.

[0049] According to the embodiment of the present invention, the automatic conference survey robot can obtain the monitoring image of the conference site through the communication component, so as to select a suitable survey object, and can move to the vicinity of the survey object, interact with the survey object through the display device, guide the survey object to participate in the survey, and improve the convenience of the survey activity. It can also analyze the enthusiasm and seriousness of the survey object in participating in the survey based on the facial image of the survey object taken by the camera, so as to determine the reference value of the answer information filled in by the survey object, and improve the accuracy and objectivity of the survey. When screening the pending survey object, it can be determined whether the object has interactive activities with other objects and whether it is suitable as a survey object through the facial orientation information of the object and the distance with other objects, thereby improving the accuracy of screening the pending survey object. When screening survey subjects, the intersection of the pending straight path and the area occupied by the subject can be used to determine whether there are obstacles on the pending straight path where the conference automatic survey robot moves to the first position information where the pending survey subject is located, and the difficulty coefficient of the conference automatic survey robot's movement is determined based on the number of obstacles. The movement difficulty score of the conference automatic survey robot to the first position information where each pending survey subject is located is comprehensively determined based on the difficulty coefficient and the length of the pending straight path, thereby improving the accuracy, objectivity and comprehensiveness of the movement difficulty score, and the pending survey subject with the lowest movement difficulty score can be selected as the survey subject, thereby improving the convenience of the survey and reducing the movement difficulty of the conference automatic survey robot. When determining the reference value score, the reference value score of the answer information input by the survey subject can be determined based on the expression category information of the survey subject when entering the answer information of each question, the time spent on entering the answer information of each question, and the score of each question, thereby comprehensively and objectively describing the reference value of the answer information.

[0050] It should be understood by those skilled in the art that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments, and the embodiments of the present invention may be deformed or modified in any way without departing from the principles.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A conference automatic research robot, characterized in that: include: Display devices, mobile components, cameras, communication components and processors; The display device is used to display the survey questionnaire according to the instructions of the processor and receive the answer information to the survey questionnaire; The moving component is used to move to a preset position around the research object according to the instruction of the processor; The camera is used to capture scene images in the conference room and send the scene images to the processor, and is also used to capture portrait images of the research subjects and send the portrait images to the processor; The communication component is used to communicate with the surveillance camera in the conference venue and obtain the surveillance images taken by the surveillance camera, and is also used to communicate with the backend server; The processor is used to: Upon receiving a survey instruction from the backend server, the monitoring images at multiple times are acquired through the communication component; Screening a candidate for investigation from among multiple objects in the conference venue according to the surveillance image; Determine, based on the monitoring image, first position information of the subject to be investigated in the conference venue and second position information of the conference automatic investigation robot in the conference venue, wherein the first position information and the second position information are spatial position information in the conference venue; selecting a research subject from among the pending research subjects according to the first location information and the second location information; According to the first position information, the second position information and the scene image captured by the camera in real time, controlling the mobile component to move to a preset position around the survey object, and displaying the survey questionnaire through the display device; At multiple moments when the research subject inputs answer information, controlling the camera to capture the research subject's facial image; Determining a reference value score of the answer information based on the facial image; The answer information and the reference value score are sent to the backend server through the communication component.

2. The automatic conference research robot according to claim 1, characterized in that: Based on the surveillance image, a candidate for investigation is screened from multiple objects in the conference venue, including: Using the trained image detection model, the locations of the objects in the surveillance image are detected to obtain a selection box for selecting each object, wherein the selection box is a minimum rectangular box for selecting the object; The image blocks selected by each selection box are processed by the trained orientation recognition model to determine the facial orientation information of each object, wherein the facial orientation information is a direction vector starting from the centroid of the facial region of the object; According to the centroid coordinates of the selection box and the facial orientation information, the subject to be determined for investigation is screened from a plurality of subjects in the conference venue.

3. The automatic conference research robot according to claim 2, characterized in that: According to the centroid coordinates of the selection box and the facial orientation information, a candidate survey subject is selected from multiple subjects in the conference venue, including: According to the formula Obtain the first screening condition C1, where: is the facial orientation information of the jth object in the ith surveillance image, is the centroid coordinate of the selection box of the jth object in the i-th monitoring image, is the centroid coordinate of the selection box of the kth object in the ith monitoring image, for The transposed vector of is the preset angle, and is the logical "and" operator, For the preset distance, is the preset number, if is the conditional function, m is the number of surveillance images at multiple moments, n is the number of objects in the surveillance images, i≤m, j≤n, k≤n, j≠k, and i, j, k, m and n are all positive integers; If the j-th object meets the first screening condition C1, the j-th object is determined as a pending research object.

4. The automatic conference research robot according to claim 1, characterized in that: Selecting a research object from the pending research objects according to the first location information and the second location information includes: Determine, according to the first position information and the second position information, a pending straight line path between the conference automatic survey robot and the first position information of each pending survey object; Determining third position information of each object in the conference venue according to the surveillance image; Determine, according to the undetermined straight line path and the third position information, a movement difficulty score of the conference automatic survey robot moving to the first position information where each undetermined survey object is located; The undetermined research object corresponding to the minimum value of the moving difficulty score is determined as the research object.

5. The automatic conference research robot according to claim 4, characterized in that: Determining the movement difficulty score of the conference automatic survey robot moving to the first position information of each to-be-determined survey object according to the to-be-determined straight line path and the third position information includes: Determining the area occupied by each object in the conference venue according to the third location information; According to the formula Determine the difficulty score of the conference automatic research robot moving to the first location information of the sth pending research object ,in, is the undetermined straight line path between the conference automatic research robot and the first position information of the sth undetermined research object, for Length, is the area occupied by the tth object, is an empty set, if is a conditional function, n is the number of objects in the monitoring image, is the preset scale factor, , t≤n, and t and n are both positive integers.

6. The automatic conference research robot according to claim 1, characterized in that: Determining a reference value score of the answer information according to the facial image includes: Determine the expression category information of the facial images at multiple moments through the trained expression detection model; A reference value score of the answer information is determined based on the expression category information.

7. The automatic conference research robot according to claim 6, characterized in that: Determining a reference value score of the answer information according to the facial image and the expression category information includes: According to the formula Determine the reference value score V of the answer information, where: The expression category information of the facial image at the tth moment within the time period when the research subject inputs the answer information of the hth question in the research questionnaire, For preset expression category information, is the score of the hth question in the survey questionnaire, is the total score of the survey questionnaire, The number of moments in the time period during which the respondents input their answers to the hth question in the survey questionnaire, is the preset time number of the hth question, H is the number of questions in the survey questionnaire, t≤ , h≤H, and t, h, and H are both positive integers.

8. The automatic conference research robot according to claim 1, characterized in that: The conference automatic research robot also includes: a multi-level storage component, which includes multiple storage spaces, each of which is used to store multiple types of gifts.

9. The automatic conference research robot according to claim 8, characterized in that: The processor is also used to: after the survey subject completes the process of inputting answer information, select a target storage space corresponding to the reference value score from multiple storage spaces of the multi-level storage component, open the target storage space, and display a prompt message for taking the gift on a display device.

Images