Information processing method and device, electronic equipment and storage medium
By eloquently identifying the dialogue text and face image information between students and patrol robots, the problem of difficulty in identifying students' negative emotions in a timely manner in the existing technology is solved, and accurate identification and early warning of students' emotions is achieved, and students' physical and mental health is promoted. Grow healthy.
Patent Information
- Application Number
- CN202311814382.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-26
- Publication Date
- 2024-11-22
AI Technical Summary
The existing technology is difficult to timely and accurately identify students' negative emotions during school learning and growth, which will affect students' physical and mental health.
By obtaining the dialogue text information and face image information between the target students and the patrol robot on campus, using pre-trained dialogue models and expression recognition technology, we can identify students' current emotional type, and decide whether to report accident warning information based on the emotion type. .
It realizes timely identification and early warning of students' negative emotions, promotes natural interaction between students and patrol robots, reflects campus humanistic care, and ensures that underage children are physically and mentally healthy during their learning and growth.
Smart Images

Figure CN119007754A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to an information processing method, device, electronic equipment and storage medium. Background Art
[0002] Nowadays, the physical and mental health of children such as kindergarteners, primary school students and middle school students is increasingly affected by the school environment in addition to the family environment. How to timely and accurately identify the negative emotions of students in the process of learning and growing in school has become a key issue that needs to be solved urgently. Summary of the invention
[0003] The present invention provides an information processing method, device, electronic device and storage medium to solve the defect that the existing system cannot timely and accurately identify students' emotions, ensure that the interaction between students and patrol robots is more natural and intimate, and also reflect the flexibility and reliability of campus humanistic care, ensuring that minors grow up healthily both physically and mentally during their learning and growth process.
[0004] The present invention provides an information processing method, comprising:
[0005] Acquire the text information of the conversation between the target student and the patrol robot on campus, and / or the facial image information of the target student;
[0006] Performing emotion recognition based on the conversation text information and / or the facial image information to determine the current emotion type of the target student;
[0007] Based on the current emotion type, determine whether to report accident warning information.
[0008] According to an information processing method provided by the present invention, the step of obtaining the conversation text information between the target student and the patrol robot on campus includes:
[0009] Acquire the voice information output by the target student to the patrol robot, and convert the voice information into text information;
[0010] Input the text information into a pre-trained dialogue model, and obtain response information output by the dialogue model that is adapted to the text information;
[0011] Based on the text information and the response information, acquiring the conversation text information;
[0012] The large dialogue model is obtained after training based on sample dialogue text information.
[0013] According to an information processing method provided by the present invention, emotion recognition is performed on the dialogue text information to determine the current emotion type, including:
[0014] The dialogue text information is segmented, and the current emotion type is determined based on the segmentation result and a mapping relationship between a preset emotion keyword and an emotion type.
[0015] According to an information processing method provided by the present invention, emotion recognition is performed on the facial image information to determine the current emotion type, including:
[0016] Performing expression recognition on the facial image information to determine the current expression type of the target student;
[0017] The current emotion type is determined based on a preset mapping relationship between expression type and emotion type.
[0018] According to an information processing method provided by the present invention, emotion recognition is performed on the conversation text information and the face image information to determine the current emotion type, including:
[0019] Determine the target emotion keywords of the target students by segmenting the dialogue text information and presetting the emotion keywords corresponding to the different emotion types;
[0020] Determine the current expression type of the target student in the process of talking with the patrol robot by performing expression recognition on the facial image information obtained during the conversation between the target student and the patrol robot;
[0021] Emotion recognition is performed on the target emotion keyword and the current expression type to determine the current emotion type.
[0022] According to an information processing method provided by the present invention, obtaining the face image information of the target student includes:
[0023] Receiving video image information uploaded by the patrol robot;
[0024] Information recognition is performed on the video image information to obtain the facial image information of the target student.
[0025] According to an information processing method provided by the present invention, determining whether to report accident warning information based on the current emotion type includes:
[0026] If it is determined that the current emotion type is a negative emotion, the accident type prediction information related to the negative emotion type is determined, and the conversation text information and / or the facial image information, as well as the accident type prediction information are reported as the accident warning information to the relevant person in charge of the school.
[0027] The present invention also provides an information processing device, comprising:
[0028] An information acquisition module, used to acquire the text information of the conversation between the target student and the patrol robot on campus, and / or the facial image information of the target student;
[0029] An information recognition module, used for performing emotion recognition based on the conversation text information and / or the face image information to determine the current emotion type of the target student;
[0030] The information processing module is used to determine whether to report accident warning information based on the current emotion type.
[0031] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the information processing methods described above is implemented.
[0032] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the information processing method described in any one of the above is implemented.
[0033] The information processing method, device, electronic device and storage medium provided by the present invention, wherein the information processing method, the server determines the current emotion type of the target student and determines whether to report the accident warning information by performing emotion recognition on the dialogue text information between the target student and the patrol robot on campus, and / or the facial image information of the target student. In this way, combined with the patrol robot collecting the dialogue information between it and the student and / or the facial image of the student in real time, it can be timely identified whether the student's current emotion is a negative emotion, so that when it is determined that the student has a negative emotion, the accident warning is reported to the relevant person in charge of the school in a timely manner, thereby ensuring that the interaction between the student and the patrol robot is more natural and intimate, and can also reflect the flexibility and reliability of campus humanistic care, and ensure that minors thrive in physical and mental health during the learning and growth process. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0035] Figure 1 It is a flowchart of the information processing method provided by the present invention;
[0036] Figure 2 It is a schematic diagram of an application scenario of the information processing method provided by the present invention;
[0037] Figure 3 It is a structural schematic diagram of the information processing device provided by the present invention;
[0038] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] Nowadays, the physical and mental health of children such as kindergarteners, primary school students and middle school students is increasingly affected by the school environment in addition to the family environment. How to timely and accurately identify the negative emotions of students in the process of learning and growing in school has become a key issue that needs to be solved urgently.
[0041] In order to solve the above technical problems, the present invention provides an information processing method, device, electronic device and storage medium, so as to realize timely and accurate identification of negative emotions of students in the process of learning and growing in school, and ensure that minors are healthier physically and mentally in the process of learning and growing.
[0042] Combine the following Figure 1-Figure 4 The information processing method, device, electronic device and storage medium of the present invention are described, wherein the execution subject of the information processing method can be an electronic device or a server, the electronic device or the server can be externally connected to a patrol robot, and communicate with the patrol robot; the patrol robot has at least a camera function, an image acquisition function, a sound pickup function, an obstacle avoidance function, a voice interaction function, a walking function, an information storage function, a data communication function, an audio and video information processing function and an image information processing function; the electronic device can be a personal computer (PC), a portable device, a laptop, a smart phone, a tablet computer and a portable wearable device and other devices cloud server, the server can refer to a server, or it can be a server cluster composed of multiple servers, a cloud server, etc. The present invention does not specifically limit the specific form of the electronic device or server. Furthermore, the information processing method can also be applied to an information processing device set in an electronic device or a server, and the information processing device can be implemented by software, hardware or a combination of the two. The following takes the execution subject of the information processing method as a server as an example to describe the information processing method.
[0043] In order to facilitate understanding of the information recommendation method provided by the embodiment of the present invention, the information recommendation method provided by the present invention will be described in detail below through the following exemplary embodiments. It can be understood that the following exemplary embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.
[0044] Reference Figure 1 , which is a flow chart of the information processing method provided by the present invention, such as Figure 1 As shown, the information processing method includes the following steps 110 to 130.
[0045] Step 110: Acquire the text information of the conversation between the target student and the patrol robot on campus, and / or the facial image information of the target student.
[0046] Among them, the patrol robot can be a robot that walks along a preset patrol route on campus; the dialogue text information can be dialogue text information generated when the target student has a dialogue with the patrol robot; the facial image information can be image information collected by the patrol robot when the target student has a dialogue with the patrol robot, or facial image information of the target student currently within the patrol robot's line of sight; the number of target students can be at least one, and each target student is a student among all the students who are learning and growing in the school.
[0047] Specifically, in step 110, the patrol robot will not only walk on the road as preset, but also first collect identity feature information of each pedestrian when it encounters at least one pedestrian head-on, at least one pedestrian appears within the image acquisition range, or at least one pedestrian appears within the video acquisition range during walking, such as collecting name tag information, height information and / or facial image information of each pedestrian, and then report the collected at least one identity feature information to the server, so that the server matches the identity feature information of each pedestrian transmitted by the patrol robot with the pre-stored identity feature information corresponding to all students in the school, and determines the pedestrian corresponding to the identity feature information collected when the match is successful as the target student; in this way, at least one target student can be determined.
[0048] At this time, for each target learning, the server can first query the historical conversation information between the target student and the patrol robot from the stored historical conversation information set. If the server does not query the historical conversation information between the target student and the patrol robot from the stored historical conversation information set, it is considered that the target student and the patrol robot have not chatted. At this time, a unique and natural current greeting text information that matches the current time and current environment can be selected from the pre-stored greeting text information library, and the identity feature information corresponding to at least one matched target student and the current greeting text information are sent to the patrol robot in language, so that the patrol robot outputs the language information corresponding to the current greeting text information to each target student, thereby bringing the patrol robot closer to each target student, thereby improving the intimacy between each target student and the patrol robot, and improving each target student's trust in the patrol robot.
[0049] For example, when the server determines a target student, if the current time is 7:30 in the morning and the current location is the school gate, it can be determined that the current greeting text message is "Good morning, remember to have breakfast"; if the current time is 12 noon and the current location is the restaurant entrance, it can be determined that the current greeting text message is "Lunch time is here, have a pleasant meal"; if the current time is 4 pm and the current weather forecast shows that it is about to rain, it can be determined that the current greeting text message is "It is about to rain, go back to the dormitory or classroom quickly, be careful not to catch a cold in the rain."
[0050] It should be noted that if the server retrieves the historical conversation information between the target student and the patrol robot from the stored historical conversation information set, the context of the retrieved historical conversation information can be understood at this time to generate greeting sentences related to the historical conversation, so that the patrol robot and the target student can continue the conversation on the previous chat topic, or generate greeting sentences that care for the target student based on the previous chat topic; the purpose of the above method is also to increase the intimacy between each target student and the patrol robot, and to increase each target student's trust in the patrol robot, so that when the target student suffers different degrees of frustration, grievances or worries, he can think of confiding in the patrol robot, and the patrol robot can also provide timely encouragement and cheering to the target student.
[0051] Step 120: Perform emotion recognition based on the conversation text information and / or the facial image information to determine the current emotion type of the target student.
[0052] Among them, the current emotion type can be the emotion type of the current target student when communicating with the patrol robot, and / or the emotion type of the target student when entering the line of sight of the patrol robot. The emotion type can include but is not limited to joy, sadness, anger, fear and other ones.
[0053] Specifically, in step 120, the server may determine the emotion type of the target student when conversing with the patrol robot by performing information recognition and context understanding on the conversation text information between the target student and the patrol robot, and / or the facial image information of the target student; it may also determine the current emotion type of the target student who is currently within the line of sight of the patrol robot by performing emotion recognition on the facial image information of the target student using pre-stored facial image information corresponding to different emotion types; it may also determine the current emotion type of the target student by performing information recognition and context understanding on the conversation text information and performing emotion recognition on the facial image information collected by the patrol robot when the target student is conversing with the patrol robot.
[0054] Step 130: Determine whether to report accident warning information based on the current emotion type of the target student.
[0055] Specifically, in step 130, when the server determines the current emotion type of the target student, it first determines whether the current emotion type is a negative emotion such as sadness or anger. If it is determined that the current emotion type is a negative emotion, the accident warning information can be reported to the relevant person in charge of the school in a timely manner; on the contrary, if it is determined that the current emotion type is not a negative emotion such as sadness or anger, for example, a positive emotion such as happiness or happiness, then no accident warning is required.
[0056] In the information processing method provided by the embodiment of the present invention, the server determines the current emotion type of the target student and determines whether to report the accident warning information by performing emotion recognition on the dialogue text information between the target student and the patrol robot on campus, and / or the facial image information of the target student. In this way, combined with the patrol robot collecting the dialogue information between it and the student and / or the facial image of the student in real time, it can be timely identified whether the student's current emotion is a negative emotion, so that when it is determined that the student has a negative emotion, the accident warning is reported to the relevant person in charge of the school in a timely manner, thereby ensuring that the interaction between the student and the patrol robot is more natural and intimate, and also reflecting the flexibility and reliability of campus humanistic care, ensuring that minors thrive in physical and mental health during the learning and growth process.
[0057] Based on the above Figure 1 In the information processing method shown in the figure, in an exemplary embodiment, in order to ensure that the conversation between the patrol robot and the student is more natural and comfortable, a large dialogue model can be used to obtain the most appropriate response information to the student's question or speech, so as to obtain the dialogue text between the student and the patrol robot. Based on this, in step 110, the server obtains the dialogue text information between the target student in the campus and the patrol robot, and the specific implementation process includes:
[0058] First, the voice information output by the target students to the patrol robot is obtained and converted into text information; the text information is further input into the pre-trained dialogue model to obtain the response information output by the dialogue model that is adapted to the text information; then, based on the text information and the response information, the dialogue text information between the target students and the patrol robot on campus is obtained.
[0059] Among them, the large dialogue model is obtained after training based on sample dialogue text information.
[0060] Specifically, the server obtains the voice information output by the target student to the patrol robot. The server can obtain the voice information by the target student actively greeting the patrol robot or generating voice information when speaking and then uploading it to the server. The server can also obtain the voice information by the target student passively replying to the patrol robot's greeting statement and generating voice information and uploading it to the server. The present invention does not specifically limit the method of obtaining the voice information.
[0061] For the acquired voice information, the server first calls a pre-stored automatic speech recognition unit to perform information recognition on the voice information, determines the text information corresponding to the converted voice information, and then inputs the text information into the pre-trained dialogue model for response information adaptation. Then, the dialogue model outputs the response information with the highest adaptability to the text information, and converts the response information into response voice information by calling a pre-stored speech synthesis unit, and sends the response voice information to the patrol robot to instruct the patrol robot to output the response voice information to the target student. Then, the server obtains the voice information output by the target student based on the response voice information through the patrol robot, and obtains the response information again through the dialogue model. In this way, the dialogue text information between the target student and the patrol robot on campus is obtained, that is, the dialogue text information includes at least one text information and the response information adapted to each text information.
[0062] It should be noted that the conversation model may be a model obtained by the server pre-acquiring a large amount of sample conversation information to train the large model, or may be an existing pre-trained conversation model. The present invention does not make specific limitations on this.
[0063] The information processing method provided by the embodiment of the present invention enables the server to obtain the most suitable way to reply to each response information of the student through the dialogue big model when the student speaks to the patrol robot, thereby stimulating the student's desire and enthusiasm for dialogue with the patrol robot, improving the comfort and naturalness of the dialogue between the student and the patrol robot, and providing reliable guarantee for the accuracy of subsequent emotion recognition.
[0064] Based on the above Figure 1In the information processing method shown, in an exemplary embodiment, considering that the patrol robot can only identify the student's emotions through dialogue when transmitting the student's speech to the server, in order to improve the accuracy of emotion recognition, the emotion keywords corresponding to different emotion types can be pre-stored so that the emotion type can be accurately identified based on the dialogue. Based on this, in step 120, the server performs emotion recognition on the dialogue text information to determine the current emotion type of the target student, and the specific implementation process may include:
[0065] The dialogue text information is segmented, and based on the segmentation result and the mapping relationship between the preset emotion keywords and emotion types, the current emotion type of the target student is determined.
[0066] Among them, emotional keywords may include but are not limited to happy, cheerful, pleasant, happy, delighted, joyful, annoyed, indignant, angry, furious, angry, displeased, furious, furious, grief, sorrow, grief, grief, sad, sorrowful, grief, painful, uncomfortable, sad, heartbroken, unhappy, sad, sorrowful, depressed, melancholy, worried, sad, fear, afraid, panic, terrified and timid, etc.
[0067] Specifically, after the server performs word segmentation on the conversation text information between the target student and the patrol robot, it can perform keyword recognition on the word segmentation results based on all emotion keywords in the pre-set mapping relationship between emotion keywords and emotion types to determine whether there are at least emotion keywords in the word segmentation structure, and determine that the emotion type corresponding to the existing emotion keywords is the current emotion type of the target student.
[0068] In the information processing method provided by the embodiment of the present invention, the server determines the current emotion type of the target student by matching the segmentation result obtained after segmenting the conversation text information with the mapping relationship between emotion keywords and emotion types. In this way, even if the server only obtains the conversation information between the student and the patrol robot, it can accurately identify the student's emotion type, thereby improving the reliability and accuracy of identifying the student's emotion.
[0069] Based on the above Figure 1 In the information processing method shown, in an exemplary embodiment, considering that the patrol robot can only recognize the student's emotions through the image when transmitting the student's face image to the server, in order to improve the accuracy of emotion recognition, the expression types corresponding to different emotion types can be pre-stored so that the emotion type can be accurately recognized based on the face image. Based on this, in step 120, the server performs emotion recognition on the face image information to determine the current emotion type of the target student, and the specific implementation process may include:
[0070] First, facial expression recognition is performed on the facial image information to determine the current expression type of the target student; and then the current emotion type of the target student is further determined based on the pre-set mapping relationship between expression type and emotion type.
[0071] Specifically, the mapping relationship between expression type and emotion type pre-stored in the server contains at least one expression identifier corresponding to each emotion type, and each expression identifier can be an expression package and / or expression outline portrait that represents a specific emotion type. Based on this, the server can perform expression recognition on the face image of the target student based on the pre-stored expression recognition model, and match the recognized current expression type with the mapping relationship between the expression type and emotion type, thereby determining the current emotion type of the target student.
[0072] It should be noted that the expression recognition model may be a model obtained by pre-acquiring a large number of sample face images by the server for expression recognition training, or may be an existing pre-trained expression recognition model. The present invention does not make specific limitations on this.
[0073] In the information processing method provided by the embodiment of the present invention, the server determines the current emotion type of the target student by matching the current expression type obtained by facial image information through expression recognition with the mapping relationship between expression type and emotion type. In this way, when the patrol robot transmits the student's facial image to the server, it can also ensure the rapid and timely identification of the student's emotion type, and can also improve the accuracy and flexibility of identifying the student's emotion.
[0074] Based on the above Figure 1 In the information processing method shown, in an exemplary embodiment, when the server obtains the chat dialogue between the patrol robot and the student and the student's face image during the chat dialogue, emotion recognition can be performed on the chat dialogue and the face image. Based on this, in step 120, the server performs emotion recognition on the dialogue text information and the face image information to determine the current emotion type of the target student. The specific implementation process may include:
[0075] Firstly, the target emotion keywords of the target students are determined by performing word segmentation on the dialogue text information and pre-setting the emotion keywords corresponding to different emotion types. Furthermore, the current expression type of the target students during the dialogue with the patrol robot is determined by performing expression recognition on the facial image information obtained during the dialogue between the target students and the patrol robot. Then, emotion recognition is performed on the target emotion keywords and the current expression type to determine the current emotion type of the target students.
[0076] Specifically, the server segments the conversation text information generated by the conversation between the patrol robot and the target student, and determines the target emotion keyword from the obtained segmentations according to at least one emotion keyword corresponding to each pre-set emotion type; at the same time, the server determines the current expression type of the target student by obtaining facial image information collected during the conversation between the patrol robot and the target student, and calling the expression recognition model to perform expression recognition on the facial image information; at this time, the server can determine the current emotion type corresponding to the target emotion keyword and the current expression type based on the mapping relationship between the pre-set emotion keyword-expression type-emotion type.
[0077] The information processing method provided by the embodiment of the present invention determines the current emotion type of the target student by using the conversation information between the patrol robot and the student and the facial images collected during the conversation to perform emotion recognition, thereby improving the accuracy and flexibility of identifying the student's emotions.
[0078] Based on the above Figure 1 In the information processing method shown, in an exemplary embodiment, when the patrol robot has a built-in camera unit, the server can receive the video image information collected by the patrol robot and perform information recognition. Based on this, in step 110, the server obtains the face image information of the target student, and the specific implementation process may include:
[0079] First, the video image information uploaded by the patrol robot is received; then the video image information is further recognized to obtain the face image information of the target student.
[0080] Specifically, when the patrol robot walks along a preset route on campus, it can collect video images of target students that appear within the camera range in real time or periodically, and upload the collected video image information to the server. The collected video image information can be multiple video frame images or a whole video formed by multiple frame images. At this time, the server can perform information recognition on the video image information, that is, recognize the video image information frame by frame, and perform facial image cropping on the identified student image information containing the target student, thereby obtaining the facial image information of the target student. Alternatively, the server performs facial image recognition on the video image information by calling a pre-built-in facial recognition model, so as to identify the facial image information of the target student from the video image information.
[0081] It should be noted that the face recognition model may be a model obtained by training the server based on a large amount of sample video image information, or may be an existing pre-trained face recognition model, which is not specifically limited here.
[0082] The information processing method provided by the embodiment of the present invention enables the server to obtain the facial image information of the target student by performing information recognition on the video image information uploaded by the patrol robot, thereby improving the convenience, speed, accuracy and reliability of obtaining the student's facial image.
[0083] Based on the above Figure 1 In the information processing method shown, in an exemplary embodiment, when the server determines that the student's emotion is negative, a corresponding accident warning can be generated in time and reported to the relevant person in charge of the school. Based on this, the specific implementation process of step 130 may include:
[0084] If it is determined that the current emotion type is a negative emotion, the accident type prediction information matching the negative emotion is determined, and the conversation text information and / or facial image information, as well as the accident type prediction information, are reported to the relevant person in charge of the school as accident warning information.
[0085] Among them, negative emotions may include but are not limited to one of other emotions such as sadness, anger, and fear; accident type prediction information may include accident prediction information that is harmful to physical and mental health caused by negative emotions.
[0086] Specifically, when the server determines that the target student's current emotion type is one of other negative emotions such as sadness, anger, fear, etc., it can be considered that the target student has negative emotions. At this time, in order to prevent the target student from having negative behavioral tendencies or negative behavioral tendencies, the server can promptly determine the accident warning information and report it to the relevant person in charge of the school in a timely manner.
[0087] The information processing method provided by the embodiment of the present invention is that when a server determines that a student has negative emotions, the server predicts the type of accident in a timely manner, determines the accident warning information, and then reports it to the relevant person in charge of the school, thereby ensuring that minors grow up healthily both physically and mentally during the learning and growth process.
[0088] For example, refer to Figure 2 The schematic diagram of the application scenario of the information processing method shown in FIG. Figure 2 In the present invention, the patrol robot is pre-equipped with a microphone unit, a camera unit and a sensor unit. The microphone unit is used to pick up the voice information of the target student during the conversation between the target student and the patrol robot, and the camera unit is used to collect video image information within the camera range; the sensor unit includes sensors for realizing the patrol robot's walking function, obstacle avoidance function and other functions, such as visual sensors, force sensors, sound sensors, tactile sensors, heat sensors, torque sensors and stress sensors.
[0089] The patrol robot transmits the voice information picked up by the microphone unit and / or the video image information collected by the camera unit to the server. The central control scheduling unit built into the server can call the automatic speech recognition (Automatic Speech RecognitionR, ASR) unit to convert the voice information into text information, and then call the dialogue model to generate response information adapted to the text information, and then call the speech synthesis (Text To Speech, TTS) unit to convert the response information into response voice information, and further transmit the response voice information to the patrol robot, and the patrol robot outputs the response voice information to the target student; in this way, the dialogue text information between the target student and the patrol robot is determined. The dialogue text information can be generated when the target student has a dialogue based on his trust in the patrol robot. The higher the trust, the more text information and response information are contained in the dialogue text information. In this way, the probability and accuracy of identifying the current emotion type from the dialogue text information are greater.
[0090] The patrol robot transmits the video image information collected by the camera unit to the server. The central control scheduling unit built into the server calls the face recognition model to identify the face image information of the target student from the video image information, and then calls the expression recognition model to perform expression recognition on the face image information to determine the current expression type of the target student, and then performs emotion recognition based on the dialogue text information and / or the current expression type to determine the current emotion type of the target student and determine whether to report the accident warning information. The specific process involved can be referred to the aforementioned embodiment. It will not be repeated here.
[0091] In this way, compared with the patrol robots in existing campuses that only have walking and obstacle avoidance functions and general interactive capabilities, the patrol robot provided by the embodiment of the present invention not only has walking and obstacle avoidance functions, but can also communicate with students, listen to students' confessions and provide timely encouragement and care, bringing warmth and care to minors, making the campus life of minors more beautiful and happier.
[0092] The information processing device provided by the present invention is described below. The information processing device described below and the information processing method described above can be referenced to each other.
[0093] Reference Figure 3 , is a schematic diagram of the structure of the information processing device provided by the present invention, such as Figure 3 As shown, the information processing device 300 includes: an information acquisition module 310, an information identification module 320 and an information processing module 330.
[0094] The information acquisition module 310 is used to acquire the conversation text information between the target student and the patrol robot on campus, and / or the facial image information of the target student.
[0095] The information recognition module 320 is used to perform emotion recognition based on the dialogue text information and / or the facial image information to determine the current emotion type of the target student.
[0096] The information processing module 330 is used to determine whether to report accident warning information based on the current emotion type.
[0097] Optionally, the information acquisition module 310 is specifically used to obtain the voice information output by the target student to the patrol robot, and convert the voice information into text information; input the text information into a pre-trained dialogue model, and obtain response information output by the dialogue model that is adapted to the text information; based on the text information and the response information, obtain the dialogue text information; wherein the dialogue model is obtained after training based on sample dialogue text information.
[0098] Optionally, the information identification module 320 is specifically configured to segment the conversation text information, and determine the current emotion type based on the segmentation result and a mapping relationship between preset emotion keywords and emotion types.
[0099] Optionally, the information recognition module 320 is specifically used to perform expression recognition on the facial image information to determine the current expression type of the target student; and determine the current emotion type based on a preset mapping relationship between expression type and emotion type.
[0100] Optionally, the information recognition module 320 is specifically used to determine the target emotion keywords of the target student by performing word segmentation on the conversation text information and pre-setting the emotion keywords corresponding to different emotion types; determine the current expression type of the target student during the conversation with the patrol robot by performing expression recognition on the facial image information obtained during the conversation between the target student and the patrol robot; and perform emotion recognition on the target emotion keywords and the current expression type to determine the current emotion type.
[0101] Optionally, the information acquisition module 310 is specifically used to receive video image information uploaded by the patrol robot; perform information recognition on the video image information, and obtain facial image information of the target student.
[0102] Optionally, the information processing module 330 is specifically used to determine the accident type prediction information related to the negative emotion type if it is determined that the current emotion type is a negative emotion, and report the conversation text information and / or facial image information, as well as the accident type prediction information as accident warning information to the relevant person in charge of the school.
[0103] The information recommendation device 300 provided in the embodiment of the present invention can execute the technical solution in any embodiment of the above-mentioned information processing method. Its implementation principle and beneficial effects are similar to those of the information processing method. Please refer to the implementation principle and beneficial effects of the information processing method, and no further description will be given here.
[0104] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communications interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the information processing method, which includes:
[0105] Acquire the conversation text information between the target student and the patrol robot on campus, and / or the facial image information of the target student; perform emotion recognition based on the conversation text information and / or the facial image information to determine the current emotion type of the target student; based on the current emotion type, determine whether to report accident warning information.
[0106] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0107] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the information processing method provided by the above methods, the method comprising:
[0108] Acquire the conversation text information between the target student and the patrol robot on campus, and / or the facial image information of the target student; perform emotion recognition based on the conversation text information and / or the facial image information to determine the current emotion type of the target student; based on the current emotion type, determine whether to report accident warning information.
[0109] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the information processing method provided by the above methods, the method comprising:
[0110] Acquire the conversation text information between the target student and the patrol robot on campus, and / or the facial image information of the target student; perform emotion recognition based on the conversation text information and / or the facial image information to determine the current emotion type of the target student; based on the current emotion type, determine whether to report accident warning information.
[0111] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An information processing method, characterized in that: include: Acquire the text information of the conversation between the target student and the patrol robot on campus, and / or the facial image information of the target student; Performing emotion recognition based on the conversation text information and / or the facial image information to determine the current emotion type of the target student; Based on the current emotion type, determine whether to report accident warning information.
2. The information processing method according to claim 1, characterized in that: The obtaining of the conversation text information between the target student and the patrol robot on campus includes: Acquire the voice information output by the target student to the patrol robot, and convert the voice information into text information; Input the text information into a pre-trained dialogue model, and obtain response information output by the dialogue model that is adapted to the text information; Based on the text information and the response information, acquiring the conversation text information; The large dialogue model is obtained after training based on sample dialogue text information.
3. The information processing method according to claim 1, characterized in that: Performing emotion recognition on the conversation text information to determine the current emotion type includes: The dialogue text information is segmented, and the current emotion type is determined based on the segmentation result and a mapping relationship between a preset emotion keyword and an emotion type.
4. The information processing method according to claim 1, characterized in that: Performing emotion recognition on the facial image information to determine the current emotion type includes: Performing expression recognition on the facial image information to determine the current expression type of the target student; The current emotion type is determined based on a preset mapping relationship between expression type and emotion type.
5. The information processing method according to claim 1, characterized in that: Performing emotion recognition on the conversation text information and the facial image information to determine the current emotion type includes: Determine the target emotion keywords of the target students by segmenting the dialogue text information and presetting the emotion keywords corresponding to the different emotion types; Determine the current expression type of the target student in the process of talking with the patrol robot by performing expression recognition on the facial image information obtained during the conversation between the target student and the patrol robot; Emotion recognition is performed on the target emotion keyword and the current expression type to determine the current emotion type.
6. The information processing method according to any one of claims 1 to 5, characterized in that: Obtaining facial image information of the target student, including: Receiving video image information uploaded by the patrol robot; Information recognition is performed on the video image information to obtain the facial image information of the target student.
7. The information processing method according to any one of claims 1 to 5, characterized in that: The determining whether to report accident warning information based on the current emotion type includes: If it is determined that the current emotion type is a negative emotion, the accident type prediction information related to the negative emotion type is determined, and the conversation text information and / or the facial image information, as well as the accident type prediction information are reported as the accident warning information to the relevant person in charge of the school.
8. An information processing device, characterized in that: include: An information acquisition module, used to acquire the text information of the conversation between the target student and the patrol robot on campus, and / or the facial image information of the target student; An information recognition module, used for performing emotion recognition based on the conversation text information and / or the face image information to determine the current emotion type of the target student; The information processing module is used to determine whether to report accident warning information based on the current emotion type.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the information processing method according to any one of claims 1 to 7 is implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the information processing method according to any one of claims 1 to 7 is implemented.
Citation Information
Cited By
Intelligent talkback terminal face recognition method based on external TF card and related equipment
CN120071452A
A face recognition method and related equipment for intelligent intercom terminal based on external TF card
CN120071452B
