Method, device and equipment for detecting emotion of person in vehicle and medium
By acquiring and adjusting the position change areas of key points of the face of passengers in the car, and performing abnormal detection combined with air quality data, the problem of low accuracy of emotion recognition in the prior art is solved, and a higher accuracy of emotion detection is achieved.
Patent Information
- Application Number
- CN202411522422.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-03-21
AI Technical Summary
In the prior art, the accuracy of emotional recognition of people in the vehicle is poor, mainly because they rely on individual face images for emotional recognition, and the impact of environmental factors on emotion detection cannot be effectively considered.
By obtaining the type and real-time location of the face key points of the target passenger, as well as the air quality data in the car, adjust the normal position change area of the face key points and perform abnormal detection to improve the accuracy of mood detection.
By considering the impact of air quality data on the location of key points on the face, adjusting the standards of mood detection to avoid interference from environmental factors, significantly improving the accuracy of emotional detection for people in the car.
Smart Images
Figure CN119672754A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent vehicle technology, and in particular to a method, device, equipment, and medium for detecting the emotions of occupants in a vehicle. Background Technology
[0002] With the development of intelligent driving technology, by recognizing the emotions of passengers, the driving experience of passengers can be determined, thereby optimizing the vehicle to improve its safety and comfort.
[0003] Most existing emotion recognition methods rely on visual information, that is, by collecting facial images of passengers and recognizing the facial images to obtain the emotion recognition results of passengers. This existing technology relies on individual facial images for emotion recognition, which has the drawback of poor accuracy in the emotion recognition results of passengers. Summary of the Invention
[0004] This invention provides a method, device, equipment, and medium for detecting the emotions of occupants in a vehicle, in order to improve the accuracy of detecting the emotions of occupants in a vehicle.
[0005] In a first aspect, the present invention provides a method for detecting the emotions of occupants in a vehicle, comprising:
[0006] Acquire the type and real-time location of at least one facial key point of the target passenger, and the air quality data inside the vehicle where the target passenger is located;
[0007] For each facial key point, obtain the normal positional variation area corresponding to the type of facial key point.
[0008] Based on the real-time location of facial key points and air quality data, the normal position change area corresponding to the type of facial key points is adjusted to obtain the adjusted position area.
[0009] Based on the type of facial key points, the adjustment area corresponding to the facial key points and the real-time position of the facial key points are used to perform anomaly detection on the facial key points and obtain the position anomaly detection results of the facial key points.
[0010] Based on the abnormality detection results of each key facial point, abnormality detection of the target passenger's emotions is performed to determine the abnormality detection results of the target passenger's emotions.
[0011] Secondly, the present invention also provides an emotion detection device for occupants in a vehicle, comprising:
[0012] The data acquisition module is used to acquire the type and real-time location of at least one facial key point of the target passenger, as well as the air quality data inside the vehicle where the target passenger is located.
[0013] The region acquisition module is used to acquire the normal positional variation region corresponding to the type of each facial key point.
[0014] The region adjustment module is used to adjust the normal position change area corresponding to the type of facial key points based on the real-time position of the facial key points and air quality data, so as to obtain the adjusted position area.
[0015] The first detection module is used to perform anomaly detection on the facial key points according to the adjustment position area corresponding to the type of facial key points and the real-time position of the facial key points, and to obtain the position anomaly detection result of the facial key points.
[0016] The second detection module is used to detect abnormalities in the emotions of the target passenger based on the abnormality detection results of the positions of each facial key point, and to determine the abnormality detection results of the target passenger's emotions.
[0017] Thirdly, embodiments of the present invention also provide an electronic device, comprising:
[0018] At least one processor; and
[0019] A memory that is communicatively connected to at least one processor; wherein
[0020] The memory stores instructions that can be executed by at least one processor, such that the at least one processor is able to perform the emotion detection method for occupants in a vehicle provided in any embodiment of the present invention.
[0021] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute the emotion detection method for occupants in a vehicle according to any embodiment of the present invention.
[0022] The technical solution of this invention involves acquiring the type and real-time location of at least one facial key point of a target passenger, as well as air quality data inside the vehicle. For each facial key point, a normal position change area corresponding to its type is acquired. Based on the real-time location of the facial key point and the air quality data, the normal position change area corresponding to the type of facial key point is adjusted to obtain an adjusted position area. Based on the adjusted position area corresponding to the type of facial key point and the real-time location of the facial key point, anomaly detection is performed on the facial key point to obtain anomaly detection results. Based on the anomaly detection results of each facial key point, anomaly detection of the target passenger's emotions is performed to determine the anomaly detection results of the target passenger's emotions. The normal position change area corresponding to each type of facial key point can be adjusted and corrected based on the air quality data inside the vehicle, thereby obtaining normal position change areas corresponding to each type of facial key point that conform to the current air quality environment. By using the real-time location of each facial key point of the target passenger and the adjusted normal position change area corresponding to each type of facial key point, the influence of air quality on the emotion detection of passengers inside the vehicle is avoided, improving the accuracy of emotion detection.
[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a flowchart of a method for detecting the emotions of occupants in a vehicle, according to Embodiment 1 of the present invention.
[0026] Figure 2 This is a flowchart of a method for detecting the emotions of occupants in a vehicle, according to Embodiment 2 of the present invention.
[0027] Figure 3 This is a flowchart of a method for detecting the emotions of occupants in a vehicle, according to Embodiment 3 of the present invention;
[0028] Figure 4 This is a schematic diagram of the structure of an emotion detection device for occupants in a vehicle according to Embodiment 4 of the present invention;
[0029] Figure 5This is a schematic diagram of the structure of an electronic device that implements the emotion detection method for occupants in a vehicle according to an embodiment of the present invention. Detailed Implementation
[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0031] It should be noted that the terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0032] The acquisition, storage, and application of real-time location and air quality data involved in the technical solutions of this invention comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0033] Example 1
[0034] Figure 1 This is a flowchart of a method for detecting the emotions of people inside a vehicle, provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where emotions of people inside a vehicle are detected. The method can be executed by an emotion detection device for people inside a vehicle. The emotion detection device for people inside a vehicle can be implemented in hardware and / or software and specifically configured in an electronic device, such as a server.
[0035] See Figure 1 The methods for detecting the emotions of occupants in the vehicle, as shown, include:
[0036] S101. Obtain the type and real-time location of at least one facial key point of the target passenger, and the air quality data inside the vehicle where the target passenger is located.
[0037] In this embodiment, the target passenger can be the passenger undergoing emotion detection. The types of facial key points can include, but are not limited to, the left and right corners of the eyes, pupils, eyebrows, nose tip, left and right corners of the mouth, etc. It should be noted that the types of facial key points listed above are merely examples, and this invention does not limit the types of facial key points. The air quality data can be the air quality data inside the vehicle where the target passenger is located.
[0038] S102. For each facial key point, obtain the normal position change area corresponding to the type of facial key point.
[0039] In this embodiment, the normal position change area can be the area where the position of facial key points changes even if the emotion abnormality detection result is normal.
[0040] S103. Based on the real-time position of facial key points and air quality data, adjust the normal position change area corresponding to the type of facial key points to obtain the adjusted position area.
[0041] In this embodiment, the adjusted position area can be the area obtained by adjusting the normal position change area. Specifically, a certain algorithm is used to adjust the normal position change area corresponding to the type of facial key points based on the real-time position of the facial key points and air quality data, thus obtaining the adjusted position area.
[0042] S104. Based on the type of facial key points, adjust the position area accordingly and the real-time position of the facial key points, perform anomaly detection on the facial key points, and obtain the position anomaly detection result of the facial key points.
[0043] Specifically, for each facial key point, if the real-time position of the facial key point is located in the adjustment position area corresponding to the type of the facial key point, then the position anomaly detection result of the facial key point is determined to be normal; otherwise, the position anomaly detection result of the facial key point is determined to be abnormal.
[0044] S105. Based on the abnormality detection results of each facial key point, perform abnormality detection on the target passenger's emotions and determine the abnormality detection results of the target passenger's emotions.
[0045] Specifically, a certain algorithm is used to detect abnormalities in the emotions of the target passenger based on the results of abnormality detection of the position of each key facial point, and to determine the abnormality detection results of the target passenger's emotions.
[0046] Optionally, based on the positional anomaly detection results of each facial key point, the target passenger's emotions are detected for anomalies, and the result of the abnormal emotion detection of the target passenger is determined. This includes: counting the first number of facial key points with abnormal positional anomaly detection results and the second number of facial key points with normal positional anomaly detection results; comparing the first number and the second number to obtain a comparison result; and determining the result of the abnormal emotion detection of the target passenger based on the comparison result.
[0047] Specifically, if the first number is greater than or equal to the second number, the emotional abnormality detection result of the target passenger is determined to be abnormal; if the first number is less than the second number, the emotional abnormality detection result of the target passenger is determined to be normal.
[0048] Understandably, by adopting the above technical solution, the detection results of abnormal emotions of the target passenger can be determined by comparing the number of facial key points in abnormal positions with the number of facial key points in normal positions, thus improving the accuracy of the determined abnormal emotion detection results.
[0049] The technical solution of this invention involves acquiring the type and real-time location of at least one facial key point of a target passenger, as well as air quality data inside the vehicle. For each facial key point, a normal position change area corresponding to its type is acquired. Based on the real-time location of the facial key point and the air quality data, the normal position change area corresponding to the type of facial key point is adjusted to obtain an adjusted position area. Based on the adjusted position area corresponding to the type of facial key point and the real-time location of the facial key point, anomaly detection is performed on the facial key point to obtain anomaly detection results. Based on the anomaly detection results of each facial key point, anomaly detection of the target passenger's emotions is performed to determine the anomaly detection results of the target passenger's emotions. The normal position change area corresponding to each type of facial key point can be adjusted and corrected based on the air quality data inside the vehicle, thereby obtaining normal position change areas corresponding to each type of facial key point that conform to the current air quality environment. By using the real-time location of each facial key point of the target passenger and the adjusted normal position change area corresponding to each type of facial key point, the influence of air quality on the emotion detection of passengers inside the vehicle is avoided, improving the accuracy of emotion detection.
[0050] Example 2
[0051] Figure 2 This is a flowchart of a method for detecting the emotions of people inside a vehicle, provided in Embodiment 2 of the present invention. Based on the technical solution of the above embodiments, the present invention optimizes and improves the adjustment operation of the normal position change area.
[0052] Furthermore, the process of "adjusting the normal position change area corresponding to the type of facial key points based on the real-time position of facial key points and air quality data to obtain the adjusted position area" is refined into "finding matching position points in each air detection point whose position is the same as the real-time position of the facial key points; querying the position adjustment coefficient that matches the real-time position and air detection data based on the real-time position of the facial key points and the air detection data of the matching position points; adjusting the normal position change area corresponding to the type of facial key points based on the position adjustment coefficient to obtain the adjusted position area," in order to improve the adjustment operation of the normal position change area.
[0053] It should be noted that for any parts not described in detail in the embodiments of the present invention, please refer to the description in the foregoing embodiments.
[0054] See Figure 2 The methods for detecting the emotions of occupants in the vehicle, as shown, include:
[0055] S201. Obtain the type and real-time location of at least one facial key point of the target passenger, and the air quality data inside the vehicle where the target passenger is located.
[0056] In this embodiment, the air quality data may include, but is not limited to, air detection data from at least one air monitoring point and the location of each air monitoring point. Here, an air monitoring point can be a location used to detect air quality; air detection data can be data obtained by detecting air quality; and the locations of each air monitoring point are different.
[0057] S202. For each facial key point, obtain the normal position change area corresponding to the type of facial key point.
[0058] S203. Among each air detection point, find a matching location point whose location is the same as the real-time location of the facial key point.
[0059] In this embodiment, the matching location point can be an air detection point whose location is the same as the real-time location of the facial key point. Specifically, based on the location of each air detection point, an air detection point whose location is the same as the real-time location of the facial key point is found, and the found air detection point is determined as the matching location point.
[0060] S204. Based on the real-time location of facial key points and the air detection data of matching location points, query the position adjustment coefficient that matches the real-time location and air detection data.
[0061] In this embodiment, the position adjustment coefficient can be used to adjust areas of normal position change. Specifically, a certain algorithm is used to query the position adjustment coefficient that matches the real-time position of facial key points and the air detection data of matching position points.
[0062] Optionally, the air detection data includes pollutant concentration data for at least one pollutant; the pollutant concentration data includes real-time concentration data and historical smoothed concentration data; the historical smoothed concentration data is historical concentration data that has undergone smoothing processing; the detection time of the historical concentration data is before the detection time of the real-time concentration data, and the detection time of the historical concentration data is adjacent to the detection time of the real-time concentration data; before querying the position adjustment coefficient matching the real-time position and air detection data based on the real-time position of the facial key points and the air detection data of the matching position points, the method further includes: for each matching position point, smoothing the real-time concentration data of each pollutant at the matching position point based on the historical smoothed concentration data of each pollutant at the matching position point, to obtain the real-time smoothed concentration data of each pollutant at the matching position point; for each pollutant, normalizing the real-time smoothed concentration data of the pollutant at each matching position point, to obtain the normalized concentration data of the pollutant at each matching position point; and updating the real-time concentration data of the pollutant at each matching position point based on the normalized concentration data of the pollutant at each matching key point.
[0063] Pollutants may include, but are not limited to, carbon dioxide, volatile organic compounds (VOCs), and fine particulate matter (PM2.5). Real-time concentration data can be concentration data obtained through real-time detection. Real-time smoothed concentration data can be concentration data obtained by smoothing real-time concentration data; normalized concentration data can be concentration data obtained by normalizing real-time smoothed concentration data.
[0064] Specifically, for each matched location point, based on the historical smoothed concentration data of each pollutant at that location point, the real-time concentration data at that location point is smoothed to obtain the real-time smoothed concentration data of each pollutant at the matched location point. For example, the real-time smoothed concentration data can be determined using the following formula:
[0065] S i，j，t =α×V i，j，t +(1-α)×S i，j，t-1 ;
[0066] Among them, S i，t This represents the real-time smoothed concentration data of the i-th pollutant at the j-th matching location point at time T; α represents the smoothing coefficient; V i，j，t S represents the real-time concentration data of the i-th pollutant at the j-th matching location point at time t; i，j，t-1This represents the historical smoothed concentration data of the i-th pollutant at the j-th matching location point at time t-1. It should be noted that the smoothing coefficient can be set independently by technicians based on actual needs or practical experience; this invention does not impose any limitations on this. The process for determining the historical smoothed concentration data is similar to the process for determining the real-time smoothed concentration data, and will not be elaborated here.
[0067] For each pollutant, determine the maximum and minimum values of its real-time smoothed concentration data at each spatial detection point; based on these values, normalize the real-time smoothed concentration data of the pollutant at each matched location point to obtain the normalized concentration data of the pollutant at each matched location point; update the real-time concentration data of the pollutant at each matched location point with the corresponding normalized concentration data; for example, the normalized concentration data can be determined using the following formula:
[0068]
[0069] Where, N i，j，t This represents the normalized concentration data of the i-th pollutant at time t at the j-th matching location; I i，t A represents the minimum value among the real-time smoothed concentration data of the i-th pollutant at time t, across all matched locations; i，t Let represent the maximum value among the real-time smoothed concentration data of the i-th pollutant at time t, for each matched location.
[0070] Understandably, by adopting the above technical solution, smoothing the real-time concentration data in the time dimension can eliminate error values in the real-time concentration data and improve the accuracy of the concentration data; in the spatial dimension, normalizing the real-time smoothed concentration data of each matching spatial point can scale the real-time smoothed concentration data of each matching spatial point to the same scale, reduce the magnitude difference between each real-time smoothed concentration data, and thus improve the accuracy of the found location adjustment coefficient.
[0071] S205. Based on the position adjustment coefficient, adjust the normal position change area corresponding to the type of facial key points to obtain the adjusted position area.
[0072] Specifically, a certain algorithm is used to adjust the normal position change area corresponding to the type of facial key points based on the position adjustment coefficient, so as to obtain the adjusted position area.
[0073] Optionally, the position adjustment coefficient includes a horizontal adjustment coefficient and a vertical adjustment coefficient; based on the position adjustment coefficient, the normal position change area corresponding to the type of facial key point is adjusted to obtain the adjusted position area, including: adjusting the normal position change area in the horizontal direction based on the horizontal adjustment coefficient, and adjusting the normal position change area in the vertical direction based on the vertical adjustment coefficient to obtain the adjusted position area.
[0074] The horizontal adjustment coefficient can be used to adjust the horizontal length of the area of normal position change; the vertical adjustment coefficient can be used to adjust the vertical height of the area of normal position change. The adjustment area can be the adjusted area of normal position change.
[0075] Specifically, the horizontal length of the area of normal position change is adjusted according to the horizontal adjustment coefficient, and the vertical height of the area of normal position change is adjusted according to the vertical adjustment coefficient to obtain the adjusted position area.
[0076] Understandably, by adopting the above technical solution, the normal position change area can be adjusted in both the horizontal and vertical directions, thus improving the accuracy of the position adjustment area.
[0077] S206. Based on the type of facial key points, adjust the position area accordingly and the real-time position of the facial key points, perform anomaly detection on the facial key points, and obtain the position anomaly detection result of the facial key points.
[0078] S207. Based on the abnormality detection results of each facial key point, perform abnormality detection on the target passenger's emotions and determine the abnormality detection results of the target passenger's emotions.
[0079] This invention, through various air detection points, finds matching locations that match the real-time location of facial key points. Based on the real-time location of the facial key points and the air detection data of the matching locations, it queries a position adjustment coefficient that matches the real-time location and air detection data. Based on the position adjustment coefficient, it adjusts the normal position change area corresponding to the type of facial key point, obtaining an adjusted position area. This invention, by querying the position adjustment coefficient using the real-time location of the facial key points and the air detection data of the matching locations, and adjusting the normal position change area, can obtain the position change area affected by real-time air conditions. By using the position change area affected by real-time air conditions and the real-time location of the facial key points, it can perform anomaly detection on the facial key points, obtaining anomaly detection results and improving the accuracy of facial key point position anomaly detection results.
[0080] Example 3
[0081] Figure 3This is a flowchart of a method for detecting the emotions of people inside a vehicle, provided in Embodiment 3 of the present invention. The present invention has made additional optimizations based on the technical solutions of the above embodiments.
[0082] Furthermore, the method is to "acquire at least one labeled facial image; wherein the labeled facial image is a facial image with normal emotion annotation results; for each labeled facial image, identify at least one facial key point and the type of each facial key point; for each type of facial key point, determine the normal position change area corresponding to the type of facial key point based on the position of the type of facial key point in each labeled facial image", in order to improve the operation of determining the normal position change area.
[0083] It should be noted that for any parts not described in detail in the embodiments of the present invention, please refer to the description in the foregoing embodiments.
[0084] See Figure 3 The methods for detecting the emotions of occupants in the vehicle, as shown, include:
[0085] S301. Obtain at least one labeled facial image; wherein, the labeled facial image is a facial image with a normal emotion labeling result.
[0086] S302. For each labeled facial image, identify at least one facial key point in the labeled facial image and the type of each facial key point.
[0087] S303. For each type of facial key point, determine the normal position variation area corresponding to the type of facial key point based on the position of the facial key point in each labeled facial image.
[0088] Specifically, using a certain algorithm, for each type of facial key point, the normal position variation area corresponding to the type of facial key point is determined based on the position of the facial key point in each labeled facial image.
[0089] Optionally, based on the location of facial key points of different types in each labeled facial image, the normal positional variation area of facial key points is determined, including: calculating the mean position of facial key points of different types in each labeled facial image, and the standard deviation of the position of facial key points in each labeled facial image; and determining the normal positional variation area corresponding to the type of facial key point based on the mean position and the standard deviation of the position.
[0090] Specifically, for each type of facial keypoint, the mean horizontal and vertical positions of that type of facial keypoint in each labeled facial image are statistically analyzed, along with the standard deviations of the horizontal and vertical positions of that type of facial keypoint in each labeled facial image. Based on the mean and standard deviation of the horizontal position, the horizontal range of variation is determined; based on the mean and standard deviation of the vertical position, the vertical range of variation is determined; based on the horizontal and vertical ranges of variation, the corresponding normal positional variation area is determined. For example, the horizontal range of variation can be determined using the following formula:
[0091] x range =[x mean -k x ×x std y mean +k x ×x std ];
[0092] Where, x range Indicates the range of horizontal variation; x mean Indicates the mean of horizontal positions; k x The weights corresponding to the standard deviation of the horizontal position; x std Indicates the standard deviation of horizontal position;
[0093] For example, the vertical variation range can be determined using the following formula:
[0094]
[0095] Among them, y range Indicates the range of vertical variation; y mean Indicates the average value of the vertical position; k y The weights corresponding to the standard deviation of the vertical position are represented by y; std This represents the standard deviation of the vertical position.
[0096] It should be noted that the weights corresponding to the standard deviation of the horizontal position and the standard deviation of the vertical position can be set independently by technical personnel according to actual needs or practical experience, and this invention does not limit this.
[0097] Understandably, by employing the above technical solution, the average position of facial key points of different types in each labeled facial image is statistically analyzed, along with the standard deviation of their positions in each labeled facial image. Based on the average and standard deviation, the normal position variation area corresponding to the type of facial key point is determined. By using the average and standard deviation, the changes in facial key points in the horizontal and vertical directions under normal emotional conditions can be determined, thereby identifying the normal position variation area and improving the accuracy of the normal position variation area.
[0098] It should be noted that steps S301, S302, and S303 are executed only once. They are used to predetermine the normal position change area corresponding to the type of facial key point before obtaining the type and real-time location of at least one facial key point of the target passenger and the air quality data inside the vehicle where the target passenger is located. This is to facilitate the acquisition of the determined normal position change area, thereby improving the efficiency of emotion detection for the passenger.
[0099] S304. Obtain the type and real-time location of at least one facial key point of the target passenger, and the air quality data inside the vehicle where the target passenger is located.
[0100] S305. For each facial key point, obtain the normal positional change area corresponding to the type of facial key point.
[0101] S306. Based on the real-time position of facial key points and air quality data, adjust the normal position change area corresponding to the type of facial key points to obtain the adjusted position area.
[0102] S307. Based on the type of facial key points, adjust the position area accordingly and the real-time position of the facial key points, perform anomaly detection on the facial key points, and obtain the position anomaly detection result of the facial key points.
[0103] S308. Based on the abnormality detection results of the positions of each facial key point, perform abnormality detection on the emotions of the target passenger and determine the abnormality detection results of the emotions of the target passenger.
[0104] This invention improves the accuracy of normal position variation regions by acquiring at least one labeled facial image, wherein the labeled facial image is a facial image with normal emotion annotation results; for each labeled facial image, at least one facial key point and the type of each facial key point are identified; for each type of facial key point, the normal position variation region corresponding to the type of facial key point is determined according to the position of the type of facial key point in each labeled facial image.
[0105] Example 4
[0106] Figure 4 This is a schematic diagram of a vehicle occupant emotion detection device according to Embodiment 4 of the present invention. This embodiment is applicable to situations involving emotion detection of vehicle occupants. The device can execute an emotion detection method for vehicle occupants and can be implemented in hardware and / or software. The device can be configured in an electronic device.
[0107] See Figure 4The vehicle occupant emotion detection device shown includes a data acquisition module 401, a region acquisition module 402, a first detection module 404, and a second detection module 405, wherein...
[0108] The data acquisition module 401 is used to acquire the type and real-time location of at least one facial key point of the target passenger, and the air quality data inside the vehicle where the target passenger is located;
[0109] The region acquisition module 402 is used to acquire the normal position change region corresponding to the type of each facial key point.
[0110] The region acquisition module 402 is used to adjust the normal position change region corresponding to the type of facial key points based on the real-time position of the facial key points and air quality data, so as to obtain the adjusted position region.
[0111] The first detection module 404 is used to perform anomaly detection on the facial key points according to the adjustment position area corresponding to the type of facial key points and the real-time position of the facial key points, and to obtain the position anomaly detection result of the facial key points.
[0112] The second detection module 405 is used to detect abnormalities in the emotions of the target passenger based on the abnormality detection results of the positions of each facial key point, and to determine the abnormality detection results of the target passenger's emotions.
[0113] This invention employs a data acquisition module to acquire the type and real-time location of at least one facial key point of a target passenger, as well as air quality data inside the vehicle where the target passenger is located; a region acquisition module to acquire the normal position change region corresponding to the type of each facial key point; a region adjustment module to adjust the normal position change region corresponding to the type of the facial key point based on the real-time location of the facial key point and the air quality data, thereby obtaining an adjusted position region; and a first detection module to perform anomaly detection on the facial key points based on the adjusted position region corresponding to the type of the facial key point and the real-time location of the facial key points, thereby obtaining a facial key point. The system detects anomalies in the position of facial key points. A second detection module is used to detect anomalies in the target passenger's emotions based on these anomaly detection results. The system can adjust and correct the normal position change areas corresponding to each type of facial key point based on in-vehicle air quality data, thus obtaining normal position change areas for each type of facial key point that conform to the current air quality environment. By using the real-time position of each facial key point of the target passenger and the adjusted normal position change areas for each type of facial key point, the system avoids the influence of air quality on the detection of emotions in the vehicle, improving the accuracy of emotion detection.
[0114] Optionally, the air quality data includes air monitoring data from at least one air monitoring point and the location of each air monitoring point;
[0115] The region adjustment module 403 includes:
[0116] The location point query unit is used to find matching location points among various air detection points whose locations are the same as the real-time locations of facial key points;
[0117] The coefficient query unit is used to query the position adjustment coefficient that matches the real-time position and air detection data of the facial key points based on the real-time position and the air detection data of the matching position points.
[0118] The region adjustment unit is used to adjust the normal position change area corresponding to the type of facial key points according to the position adjustment coefficient, so as to obtain the adjusted position area.
[0119] Optionally, the air detection data includes pollutant concentration data for at least one pollutant; the pollutant concentration data includes real-time concentration data and historical smoothed concentration data; the historical smoothed concentration data is historical concentration data that has undergone smoothing processing; the detection time of the historical concentration data is before the detection time of the real-time concentration data, and the detection time of the historical concentration data is adjacent to the detection time of the real-time concentration data.
[0120] The device also includes:
[0121] The smoothing module is used to smooth the real-time concentration data of each pollutant at each matching location point based on the historical smoothed concentration data of each pollutant at the matching location point, so as to obtain the real-time smoothed concentration data of each pollutant at the matching location point.
[0122] The normalization module is used to normalize the real-time smoothed concentration data of each pollutant at each matched location point for each pollutant, so as to obtain the normalized concentration data of the pollutant at each matched location point.
[0123] The update module is used to update the real-time concentration data of pollutants at each matching key point based on the normalized concentration data of pollutants at each matching key point.
[0124] Optionally, the position adjustment factor includes a horizontal adjustment factor and a vertical adjustment factor;
[0125] Based on the position adjustment coefficient, the normal position change area corresponding to the type of facial key points is adjusted to obtain the adjusted position area, including:
[0126] Based on the horizontal adjustment coefficient, the area of normal position change is adjusted in the horizontal direction, and based on the vertical adjustment coefficient, the area of normal position change is adjusted in the vertical direction to obtain the adjusted position area.
[0127] Optionally, the device may also include:
[0128] The labeled image acquisition module is used to acquire at least one labeled facial image; wherein, the labeled facial image is a facial image with a normal emotion annotation result;
[0129] The key point recognition module is used to identify at least one facial key point and the type of each facial key point in each labeled facial image.
[0130] The region determination module is used to determine the normal position variation region corresponding to the type of facial key point based on the position of the facial key point in each labeled facial image.
[0131] Optional, the region determination module includes:
[0132] The mean value unit is used to calculate the mean position of facial key points of a certain type in each labeled facial image, as well as the standard deviation of the position of facial key points in each labeled facial image.
[0133] The region determination unit is used to determine the normal positional variation region corresponding to the type of facial key points based on the positional mean and positional standard deviation.
[0134] Optionally, the second detection module includes:
[0135] The quantity statistics unit is used to count the first number of facial key points whose positional anomaly detection result is abnormal, and the second number of facial key points whose positional anomaly detection result is normal.
[0136] The comparison unit is used to compare the first quantity and the second quantity to obtain the comparison result.
[0137] The detection unit is used to determine the result of abnormal emotional state detection of the target passenger based on the comparison results.
[0138] The in-vehicle occupant emotion detection device provided in this embodiment of the invention can execute the in-vehicle occupant emotion detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the in-vehicle occupant emotion detection method.
[0139] Example 5
[0140] Figure 5A schematic diagram of an electronic device 500 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0141] like Figure 5 As shown, the electronic device 500 includes at least one processor 501 and a memory, such as a read-only memory (ROM) 502 or a random access memory (RAM) 503, communicatively connected to the at least one processor 501. The memory stores computer programs executable by the at least one processor. The processor 501 can perform various appropriate actions and processes based on the computer program stored in the ROM 502 or loaded into the RAM 503 from storage unit 508. The RAM 503 can also store various programs and data required for the operation of the electronic device 500. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0142] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0143] Processor 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 501 performs the various methods and processes described above, such as methods for detecting the emotions of occupants in a vehicle.
[0144] In some embodiments, the occupant emotion detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded into and / or installed on electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by processor 501, one or more steps of the occupant emotion detection method described above may be performed. Alternatively, in other embodiments, processor 501 may be configured to perform the occupant emotion detection method by any other suitable means (e.g., by means of firmware).
[0145] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0146] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable in-vehicle occupant emotion detection device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0147] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0149] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0150] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.
[0151] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0152] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A method for detecting emotions of people in a vehicle, characterized in that: The method comprises: Acquire the type and real-time position of at least one facial key point of the target passenger and the air quality data in the vehicle where the target passenger is located; For each of the facial key points, obtaining a normal position change area corresponding to the type of the facial key point; According to the real-time positions of the facial key points and the air quality data, the normal position change area corresponding to the type of the facial key points is adjusted to obtain an adjusted position area; According to the adjustment position area corresponding to the type of the facial key point and the real-time position of the facial key point, the facial key point is subjected to abnormality detection to obtain a position abnormality detection result of the facial key point; According to the position anomaly detection results of each of the facial key points, anomaly detection is performed on the target passenger's emotions to determine the abnormality detection results of the target passenger's emotions.
2. The method according to claim 1, characterized in that The air quality data includes air detection data of at least one air detection point and the position of each of the air detection points; The adjusting the normal position change area corresponding to the type of the facial key point according to the real-time position of the facial key point and the air quality data to obtain the adjusted position area includes: In each of the air detection points, searching for a matching position point having the same position as the real-time position of the facial key point; According to the real-time position of the facial key point and the air detection data of the matching position point, querying the position adjustment coefficient that matches the real-time position and the air detection data; According to the position adjustment coefficient, the normal position change area corresponding to the type of the facial key point is adjusted to obtain an adjusted position area.
3. The method according to claim 2, characterized in that The air detection data includes pollutant concentration data of at least one pollutant; the pollutant concentration data includes real-time concentration data and historical smoothed concentration data; the historical smoothed concentration data is historical concentration data that has been smoothed; the detection time of the historical concentration data is before the detection time of the real-time concentration data, and the detection time of the historical smoothed concentration data is adjacent to the detection time of the real-time concentration data; Before searching for a position adjustment coefficient matching the real-time position and the air detection data according to the real-time position of the facial key point and the air detection data of the matching position point, the method further includes: For each matching position point, according to the historical smoothed concentration data of each pollutant at the matching position point, the real-time concentration data of each pollutant at the matching position point is smoothed to obtain the real-time smoothed concentration data of each pollutant at the matching position point; For each pollutant, normalizing the real-time smoothed concentration data of the pollutant at each matching position point to obtain normalized concentration data of the pollutant at each matching position point; According to the normalized concentration data of the pollutant at each of the matching key points, the real-time concentration data of the pollutant at each of the matching position points is updated respectively.
4. The method according to claim 2, characterized in that: The position adjustment coefficient includes a horizontal adjustment coefficient and a vertical adjustment coefficient; The step of adjusting the normal position change region corresponding to the type of the facial key point according to the position adjustment coefficient to obtain the adjusted position region includes: The normal position change region is adjusted in the horizontal direction according to the horizontal adjustment coefficient, and the normal position change region is adjusted in the vertical direction according to the vertical adjustment coefficient to obtain an adjusted position region.
5. The method according to claim 1, characterized in that The normal position change area is determined by: Acquire at least one annotated facial image; wherein the annotated facial image is a facial image with a normal emotion annotation result; For each annotated facial image, identifying at least one facial key point and a type of each facial key point in the annotated facial image and a type of each facial key point; For each type of facial key point, a normal position change region corresponding to the type of facial key point is determined according to the position of the facial key point of the type in each of the labeled facial images.
6. The method according to claim 5, characterized in that The determining, according to the position of the facial key point of the type in each of the labeled facial images, a normal position change area corresponding to the type of the facial key point comprises: Counting the position mean of the facial key points of the type in each of the labeled facial images, and the position standard deviation of the facial key points in each of the labeled facial images; A normal position variation region corresponding to the type of the facial key point is determined according to the position mean and the position standard deviation.
7. The method according to claim 1, characterized in that The step of performing abnormality detection on the target passenger's emotion according to the position abnormality detection result of each of the facial key points to determine the abnormality detection result of the target passenger's emotion includes: Counting, among the facial key points, a first number of the facial key points whose position abnormality detection results are abnormal, and a second number of the facial key points whose position abnormality detection results are normal; comparing the first quantity and the second quantity to obtain a comparison result; According to the comparison result, the abnormal emotion detection result of the target passenger is determined.
8. An emotion detection device for people in a car, characterized in that: The device comprises: A data acquisition module, used to acquire the type and real-time position of at least one facial key point of a target passenger, and air quality data in the vehicle where the target passenger is located; A region acquisition module, used for acquiring, for each facial key point, a normal position change region corresponding to the type of the facial key point; A region adjustment module, configured to adjust the normal position change region corresponding to the type of the facial key points according to the real-time positions of the facial key points and the air quality data, to obtain an adjusted position region; A first detection module, configured to perform abnormality detection on the facial key points according to the adjustment position area corresponding to the type of the facial key points and the real-time position of the facial key points, to obtain a position abnormality detection result of the facial key points; The second detection module is used to perform abnormality detection on the emotions of the target passenger according to the position abnormality detection results of each of the facial key points, and determine the abnormality detection results of the emotions of the target passenger.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for detecting emotions of a person in a vehicle according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for detecting emotions of a person in a vehicle according to any one of claims 1 to 7 when executed.