Seating arrangement method, device, equipment, medium and product based on emotional state

Abstract

The present invention discloses a seat arrangement method, device, equipment, medium and product based on emotional state. The method comprises: when detecting that there is a passenger boarding action outside the target vehicle, determining the external passenger facial image and the external passenger height of the external passenger, and the internal passenger facial image and the internal passenger height of the internal passenger; according to the external passenger height and the internal passenger height, and the relative position between the image detection frames of each internal passenger in the internal passenger facial image, performing image fusion on the external passenger facial image and the internal passenger facial image, and determining the passenger seat fusion image corresponding to at least one passenger seat arrangement combination mode; performing image analysis on the passenger seat fusion image, and determining the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination mode; and determining the target passenger seat arrangement combination mode according to the degree of emotional influence corresponding to each passenger seat arrangement combination mode.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a seat arrangement method, device, equipment, medium and product based on emotional state. Background Art

[0002] With the development of automobile intelligence and autonomous driving technology, the emotional state of passengers in the car has become an important factor affecting the driving experience and safety. At present, when a new passenger gets on the car, or the seats between the original passengers in the car are usually random, the emotional contagion and its impact between the passengers are not fully considered. The emotional fluctuations caused by the emotional interaction between the passengers and the driver will affect the driver's driving safety and the comfort of the passengers. Summary of the invention

[0003] The present invention provides a seat arrangement method, device, equipment, medium and product based on emotional state to improve vehicle driving safety and passenger comfort.

[0004] According to one aspect of the present invention, there is provided a method for arranging passenger seats based on emotional states, the method comprising:

[0005] When an action of a passenger boarding the target vehicle is detected outside, an external passenger facial image and an external passenger height of the external passenger are determined, and an internal passenger facial image and an internal passenger height of an internal passenger of the target vehicle are determined; the internal passenger facial image includes at least one image detection frame of the internal passenger; the internal passenger includes at least the driver;

[0006] According to the height of the external occupant and the height of the internal occupant, and the relative positions between the image detection frames of each of the internal occupants in the internal occupant facial image, the external occupant facial image and the internal occupant facial image are image-fused to determine occupant seat fusion images corresponding to at least one occupant seat arrangement and combination mode; the occupant seat fusion image includes the driver and other occupants; the other occupants are occupants other than the driver;

[0007] Performing image analysis on the passenger seat fusion image to determine the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination;

[0008] A target passenger seat arrangement and combination is determined according to the emotion impact degrees corresponding to each of the passenger seat arrangement and combination modes.

[0009] According to another aspect of the present invention, there is provided a device for arranging passenger seats based on emotional states, the device comprising:

[0010] An occupant image determination module is used to determine an external occupant's facial image and an external occupant's height when detecting an occupant's boarding action outside the target vehicle, and to determine an internal occupant's facial image and an internal occupant's height inside the target vehicle; the internal occupant's facial image includes at least one image detection frame of the internal occupant; the internal occupant includes at least the driver;

[0011] an image fusion module, configured to fuse the external occupant facial image and the internal occupant facial image according to the external occupant height and the internal occupant height, and the relative positions between the image detection frames of each of the internal occupants in the internal occupant facial image, and determine occupant seat fusion images corresponding to at least one occupant seat arrangement and combination; the occupant seat fusion images include the driver and other occupants; the other occupants are occupants other than the driver;

[0012] An image analysis module, used for performing image analysis on the passenger seat fusion image to determine the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination;

[0013] The seat arrangement determination module is used to determine the target passenger seat arrangement combination according to the emotional impact degree corresponding to each passenger seat arrangement combination.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for arranging passenger seats based on emotional state as described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for arranging passenger seats based on emotional states as described in any embodiment of the present invention when executed.

[0019] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for arranging passenger seats based on emotional states as described in any one of the embodiments of the present invention is implemented.

[0020] The above technical solution combines the height and relative position of the passengers for image fusion and analyzes the emotional state of the passengers, thereby realizing the degree of emotional influence between the driver and other passengers under different passenger seat arrangements. The optimal passenger seat arrangement is selected based on the degree of emotional influence, so that the influence of other passengers on the driver's emotions is minimized, thereby improving vehicle driving safety and passenger riding comfort.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a method for arranging passenger seats based on emotional states according to a first embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of passenger seat arrangement of a target vehicle provided according to Embodiment 1 of the present invention;

[0025] Figure 3 is a schematic diagram of an image fusion process of an internal occupant detection frame and an external occupant detection frame provided according to Embodiment 1 of the present invention;

[0026] Figure 4 is a flow chart of a method for arranging passenger seats based on emotional states according to a second embodiment of the present invention;

[0027] Figure 5 is a flow chart of a method for arranging passenger seats based on emotional states according to Embodiment 3 of the present invention;

[0028] Figure 6 is a schematic diagram of vector expression of a first connection vector and a second connection vector provided according to Embodiment 3 of the present invention;

[0029] Figure 7 is a schematic diagram of expressing an influence degree vector provided according to Embodiment 3 of the present invention;

[0030] Figure 8 This is a schematic diagram of a method for determining a vector projection value provided in Embodiment 3 of the present invention;

[0031] Fig. 9 is a structural schematic diagram of a passenger seat arrangement device based on emotional state provided according to a fourth embodiment of the present invention;

[0032] Fig.10 It is a schematic diagram of the structure of an electronic device for implementing the passenger seat arrangement method based on emotional state according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1

[0036] Figure 1 This is a flowchart of a passenger seat arrangement method based on emotional state provided in the first embodiment of the present invention. This embodiment can be applied to reasonably allocate passenger seats to passengers of a target vehicle to reduce the impact on the driver's emotions. The method can be executed by a passenger seat arrangement device based on emotional state. The passenger seat arrangement device based on emotional state can be implemented in the form of hardware and / or software. The passenger seat arrangement device based on emotional state can be configured in an electronic device. Figure 1 As shown, the method includes:

[0037] S110. When it is detected that there is an occupant boarding the target vehicle from outside, determine the external occupant's facial image and external occupant's height, and determine the internal occupant's facial image and internal occupant's height; the internal occupant's facial image includes at least one image detection frame of the internal occupant; the internal occupants include at least the driver.

[0038] Among them, the target vehicle can be a vehicle whose passenger seat arrangement is to be optimized; the method for detecting the boarding action of external passengers of the target vehicle can be to perform personnel detection on passengers within a preset range of the target vehicle through a personnel detection sensor deployed outside the target vehicle; or, it is also possible to perform image acquisition of external passengers through an image acquisition device deployed outside the target vehicle, perform image analysis on the acquired images of external passengers, and determine whether the external passengers have any boarding action.

[0039] Wherein, image acquisition devices or apparatuses for acquiring images are installed on the exterior and interior of the target vehicle respectively; the image acquisition device on the exterior of the vehicle is used to acquire facial images of external occupants. Wherein, the external occupants may be persons who are about to enter the target vehicle, and the internal occupants may be persons inside the target vehicle, and the internal occupants include at least the driver. The image acquisition device or apparatus inside the vehicle is used to acquire facial images of internal occupants.

[0040] The external occupant height and the internal occupant height can be determined by collecting and determining the height of the occupant through a preset distance sensor, for example, the preset distance sensor can be an ultrasonic sensor or a laser distance sensor. Alternatively, the height of the occupant can be determined by an image acquisition device and a computer vision analysis algorithm, specifically, the occupant image can be acquired through an image acquisition device and image processing can be performed, and the occupant height can be determined by a human body detection algorithm and a feature extraction algorithm.

[0041] Among them, the facial image of the external occupant can be acquired by the external image acquisition device of the target vehicle, and the facial image of the external occupant at least includes the facial area of ​​the external occupant; the facial image of the internal occupant can be acquired by the internal image acquisition device of the target vehicle, and the facial image of the internal occupant at least includes the facial area of ​​the internal occupant.

[0042] Exemplarily, when an external detection device, such as a personnel detection sensor or an image acquisition device, detects that there is an occupant boarding the target vehicle, the facial features of the external occupant are imaged to obtain an external occupant facial image of the external occupant. Also, the external occupant height of the external occupant is determined by a preset distance sensor or computer vision analysis technology. At the same time, the facial features of the internal occupant are imaged to obtain an internal occupant facial image of the internal occupant. Also, the internal occupant height of the internal occupant is determined by a preset distance sensor or computer vision analysis technology.

[0043] S120. Perform image fusion on the external occupant facial image and the internal occupant facial image based on the height of the external occupant and the height of the internal occupant, and the relative positions between the image detection frames of each internal occupant in the internal occupant facial image, and determine occupant seat fusion images corresponding to at least one occupant seat arrangement combination; the occupant seat fusion images include the driver and other occupants; the other occupants are occupants other than the driver.

[0044] like Figure 2 A schematic diagram of the passenger seat arrangement of a target vehicle is shown. It is detected that an outside passenger a has boarded the target vehicle, and the current inside passengers of the target vehicle include the driver and the inside passenger b under the rear seat B. In this scenario, under the premise that the driver remains in the main driving seat, the current passenger seat arrangement combination can include 12 types, and the co-pilot seat is represented by the letter "Z", and the 12 types are: outside passenger a sits in the rear seat A, inside passenger b sits in the rear seat B (abbreviated as aA&b-B), aA&b-C, aA&b-Z, aB&b-A, aB&b-C, aB&b-Z, aC&b-A, aC&b-B, aC&b-Z, aZ&b-A, aZ&b-B and aZ&b-C.

[0045] It should be noted that since the relative positions of external occupants and internal occupants in the target vehicle are related to the height of the occupants themselves, in addition to the relative positions of the image detection frames, the height of the occupants is also considered during the image fusion process. Figure 3 A schematic diagram of the image fusion process of an internal occupant detection frame and an external occupant detection frame is shown. Taking the image fusion process corresponding to any of the occupant seat arrangement combinations as an example, the vertical position of the image detection frame corresponding to the external occupant in the synthesized occupant seat fusion image is determined according to the occupant heights of the internal occupants and the external occupants. According to the position area under the corresponding seat simulated by the external occupant and the relative positions between the image detection frames, the vertical and horizontal positions of the image detection frame corresponding to the external occupant in the synthesized occupant seat fusion image can be determined.

[0046] The image detection frame may be a regional detection frame obtained by performing regional positioning on the occupant's facial position during the image recognition process on the occupant's facial image.

[0047] Exemplarily, during the image fusion process, the image detection frame of the external occupant image is specifically fused with the image detection frame of the internal occupant image, so that the fusion image of the occupant seat includes both the image detection frame corresponding to the internal occupant and the image detection frame corresponding to the external occupant. The position area of ​​the image detection frame of the fusion process in the fusion image is locally fine-tuned in the vertical direction according to the height of the external occupant and the height of the internal occupant; the position area of ​​the image detection frame of the fusion process in the fusion image is locally fine-tuned in the horizontal direction according to the relative position between the image detection frames of the internal occupants. Generate corresponding fusion images of the occupant seats under different occupant seat arrangement combinations. Among them, the fusion image of the occupant seat includes at least the driver and other occupants. Among them, other occupants are occupants other than the driver. For example, for Figure 2 , the other passengers are passenger b inside the vehicle and passenger a outside the vehicle.

[0048] S130: Perform image analysis on the passenger seat fusion image to determine the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination.

[0049] For example, for any passenger seat fusion image, the facial features of each passenger in the passenger seat fusion image can be identified by using a pre-trained emotional state detection model to determine the emotional state of each passenger. The emotional state can be, for example, angry, funny, sad, happy, crying, etc. Combined with the distance between the image detection frame of other passengers and the image detection frame of the driver in the passenger seat fusion image, the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination is determined.

[0050] Specifically, the fusion image of the passenger seat with the emotional state information is input into the pre-trained emotional impact prediction model to obtain the emotional impact degree between the driver and other passengers output by the model. The emotional impact degree prediction model can be pre-trained by relevant technical personnel. Specifically, the fusion image of the passenger seat with the emotional state information in the historical time period is obtained, and the emotional impact degree between other passengers and the driver in the fusion image of the passenger seat is annotated, and the preset network model is trained using the annotated sample training set to obtain the emotional impact degree prediction model.

[0051] The emotional impact degree can be expressed in numerical form. For example, the emotional impact degree value between the driver and other passenger A is 0.9, and the emotional impact degree value between the driver and other passenger A is 0.1. The larger the value, the greater the emotional impact degree, and it is a positive impact or an active impact; the smaller the value, the smaller the emotional impact program. When the emotional impact degree value between the driver and other passenger A is a negative value, it means that the emotional impact between the driver and other passengers is a negative impact or a passive impact, and the larger the absolute value, the greater the negative impact degree; the smaller the absolute value, the smaller the negative impact degree.

[0052] S140: Determine a target passenger seat arrangement and combination according to the degree of emotional impact corresponding to each passenger seat arrangement and combination.

[0053] Exemplarily, for any passenger seat arrangement and combination, the emotional impact degree of the combination is determined according to the emotional impact degree between the driver and other passengers under the passenger seat arrangement and combination. Specifically, if the emotional impact degree between the driver and other passenger A in passenger seat arrangement and combination M is 0.9, the emotional impact degree between the driver and other passenger A is 0.2, and the emotional impact degree between the driver and other passenger C is -0.5, then the emotional impact degree of the passenger seat arrangement and combination can be determined to be 0.6. If the emotional impact degree of passenger seat arrangement and combination N is -0.3, and the emotional impact degree of passenger seat arrangement and combination Q is 0.1, then the target passenger seat arrangement is determined to be passenger seat arrangement and combination M, because the emotional impact of arrangement M is positive, and the positive impact degree is greater than that of passenger seat arrangement and combination Q. The emotional impact degree of passenger seat arrangement and combination N is negative, so after comparison, this combination can be eliminated.

[0054] The technical solution of the embodiment of the present invention determines the external passenger facial image and the external passenger height of the external passenger, and determines the internal passenger facial image and the internal passenger height of the internal passenger of the target vehicle when detecting the passenger boarding action outside the target vehicle, and fuses the external passenger facial image and the internal passenger facial image according to the external passenger height and the internal passenger height, and the relative position between the image detection frames of each internal passenger in the internal passenger facial image, and determines the passenger seat fusion image corresponding to at least one passenger seat arrangement combination mode; performs image analysis on the passenger seat fusion image to determine the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination mode; and determines the target passenger seat arrangement combination mode according to the degree of emotional influence corresponding to each passenger seat arrangement combination mode. The above technical solution realizes the degree of emotional influence between the driver and other passengers under different passenger seat arrangement combinations by combining the passenger height and relative position for image fusion and analyzing the emotional state of the passenger, and selects the optimal passenger seat arrangement combination according to the degree of emotional influence, so that the degree of emotional influence of other passengers on the driver is minimized, thereby improving the vehicle driving safety and the passenger riding comfort.

[0055] Embodiment 2

[0056] Figure 4 A flowchart of a method for arranging passenger seats based on emotional state is provided in the second embodiment of the present invention. This embodiment is optimized and improved on the basis of the above-mentioned technical solutions.

[0057] Furthermore, the step of "performing image analysis on the passenger seat fusion image to determine the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination" is refined into "inputting the passenger seat fusion image into a pre-trained emotional state prediction model to obtain the emotional states and emotional levels of the driver and other passengers output by the model; determining the degree of emotional influence between the driver and other passengers in the fusion image of each passenger seat according to the emotional states and emotional levels of the driver and other passengers, and the detection frame positions of the image detection frames corresponding to the driver and other passengers in the fusion image of each passenger position; determining the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination according to the degree of emotional influence between the driver and other passengers in the corresponding passenger seat fusion image." so as to improve the method for determining the degree of emotional influence.

[0058] It should be noted that for the parts not described in detail in the embodiments of the present invention, reference can be made to the descriptions of other embodiments. Figure 4 As shown, the method comprises the following specific steps:

[0059] S210. When it is detected that there is an occupant boarding the target vehicle from outside, determine the external occupant's facial image and external occupant's height, and determine the internal occupant's facial image and internal occupant's height; the internal occupant's facial image includes at least one image detection frame of the internal occupant; the internal occupants include at least the driver.

[0060] S220. Perform image fusion on the external occupant facial image and the internal occupant facial image based on the height of the external occupant and the height of the internal occupant, and the relative positions between the image detection frames of each internal occupant in the internal occupant facial image, and determine occupant seat fusion images corresponding to at least one occupant seat arrangement combination; the occupant seat fusion images include the driver and other occupants; the other occupants are occupants other than the driver.

[0061] S230, inputting the passenger seat fusion image into a pre-trained emotional state prediction model, and obtaining the emotional states and emotional levels of the driver and other passengers respectively output by the model.

[0062] The emotional state prediction model is used to predict the emotional state and emotional level of the occupant. The emotional state may include, for example, anger, funny, sad, furious, and disgusted. The emotional level may be the severity of the emotional state. For example, the emotional level may be divided into 1-5 levels, where 1 represents the lowest emotional intensity and 5 represents the highest emotional intensity. Different emotional states correspond to 5 emotional levels.

[0063] Exemplarily, the passenger seat fusion image is input into the emotional state prediction model, and the emotional state prediction model performs feature analysis on the facial features of the passengers under each image detection frame in the passenger seat fusion image to obtain the emotional states and emotional levels corresponding to the driver and other passengers, respectively. The emotional state prediction model may be a trained model for emotional state prediction that is already available in the prior art. To ensure the accuracy of model prediction, this embodiment also provides a model training method for an emotional state prediction model, which is more applicable and conforms to the application scenario of this application.

[0064] In an optional embodiment, the model training method of the emotional state prediction model is as follows: portrait pictures of different emotional states are obtained as a sample data set; the portrait pictures in the sample data set are labeled to obtain sample labels of the standard emotional state and the standard emotional level corresponding to each portrait picture; the sample data set with the sample labels is input into a pre-constructed network model to obtain the predicted emotional state and predicted emotional level output by the model; the network model is trained according to the standard emotional state, standard emotional level, predicted emotional state and predicted emotional level until the preset model training end conditions are met to obtain the emotional state prediction model.

[0065] The sample labeling process may specifically label the state and state level of the emotions of the portrait in the portrait image, and obtain sample labels of the standard emotional state and standard emotional level of the portrait image.

[0066] Exemplarily, according to the predicted emotional state and the standard emotional state output by the model, based on a preset loss function, a first loss value is determined; and according to the predicted emotional level and the standard emotional level output by the model, based on a preset loss function, a second loss value is determined; the first loss value and the second loss value are weighted and summed to obtain a target loss value. If the target loss value reaches a preset loss threshold, or the target loss value tends to be stable, the model training is terminated to obtain an emotional state prediction model.

[0067] S240. Determine the degree of emotional interaction between the driver and other passengers in the fused images of each passenger seat according to the emotional states and emotional levels of the driver and other passengers, and the detection frame positions of the image detection frames corresponding to the driver and other passengers in the fused images of each passenger seat.

[0068] Exemplarily, the detection frame distance between the driver and each other occupant is determined according to the detection frame positions of the image detection frames corresponding to the driver and other occupants in the fusion image of each occupant position; the longer the detection frame distance, the farther the distance between the driver and other occupants. According to the emotional state and emotional level between the driver and other occupants, a weight value is assigned to the distance between the driver and other occupants. Specifically, if the emotional state of the driver and other occupants are both positive, such as happy, joyful, etc., a positive weight is assigned, and the weight value can be set according to the emotional level, the larger the level, the larger the value, and the smaller the level, the smaller the value; if the emotional state of the driver and other occupants are both negative, such as sad, angry, etc., a negative weight is assigned, and the weight value can be set according to the emotional level; if the emotional state of the driver and other occupants is one positive and one negative, the weight positive or negative value can be determined according to the emotional level, and the specific value can be determined according to the emotional level.

[0069] Specifically, if the emotional states of driver A and other passenger B are both positive, and the driver's emotional level is level 2, and the emotional level of other passenger B is level 3, the weight value between driver A and other passenger B can be 0.5. If the emotional states of driver A and other passenger B are both negative, and the driver's emotional level is level 1, and the emotional level of other passenger B is level 2, the weight value between driver A and other passenger B can be -0.3. If the emotional state of driver A is positive, and the emotional level is level 2, and the emotional state of other passenger B is negative, and the emotional level is level 3, the weight value between driver A and other passenger B can be -0.1.

[0070] According to the distance between the driver and other passengers, based on the corresponding weight value, the degree of emotional influence between the driver and other passengers can be determined. For example, if the distance between the driver and other passengers is 3 unit lengths and the weight is 0.6, the corresponding value of the emotional influence is 1.8. The larger the positive value, the stronger the positive emotional influence; the smaller the positive value, the weaker the positive emotional influence. The larger the absolute value of the negative value, the stronger the negative emotional influence; the smaller the absolute value of the negative value, the weaker the negative emotional influence.

[0071] S250, determining the degree of emotional influence between the driver and other passengers in each passenger seat arrangement combination according to the degree of emotional influence between the driver and other passengers in the corresponding passenger seat fusion image.

[0072] Exemplarily, for any passenger seat fusion image, the degrees of emotional influence between the driver and other passengers in the image are numerically added, and the result of the addition is used as the degree of emotional influence under the passenger seat arrangement combination mode to which the passenger seat fusion image belongs.

[0073] Specifically, if the value of the emotional influence degree between driver A and other passenger B corresponding to the passenger seat fusion image is 1.8, the value of the emotional influence degree between driver A and other passenger C is -0.6, and the value of the emotional influence degree between driver A and other passenger C is 0.5, then the emotional influence degree under the passenger seat arrangement combination mode to which the passenger seat fusion image belongs is 1.7.

[0074] S260: Determine a target passenger seat arrangement and combination according to the degree of emotional impact corresponding to each passenger seat arrangement and combination.

[0075] The passenger seat arrangement and combination method with the largest value corresponding to the degree of emotional impact can be determined as the standard passenger seat arrangement and combination method.

[0076] The technical solution of this embodiment inputs the passenger seat fusion image into the pre-trained emotional state prediction model to obtain the emotional state and emotional level of the driver and other passengers output by the model, and determines the degree of emotional influence between the driver and other passengers in each passenger seat fusion image according to the emotional state and emotional level of the driver and other passengers, and the detection frame position of the image detection frame corresponding to the driver and other passengers in each passenger position fusion image; and determines the degree of emotional influence between the driver and other passengers in each passenger seat arrangement and combination mode according to the degree of emotional influence between the driver and other passengers in the corresponding passenger seat fusion image. The above technical solution realizes the accurate determination of the degree of emotional influence by considering the driver's emotional state and relative position distance in the process of determining the degree of emotional influence between the driver and other passengers, thereby improving the accuracy of determining the target passenger seat arrangement and combination mode.

[0077] Embodiment 3

[0078] Figure 5 A flowchart of a method for arranging passenger seats based on emotional state is provided in Embodiment 3 of the present invention. This embodiment is optimized and improved on the basis of the above-mentioned technical solutions.

[0079] Furthermore, the step of "determining the degree of emotional influence between the driver and other passengers in each passenger seat fusion image according to the emotional states and emotional levels respectively corresponding to the driver and other passengers, and the detection frame positions of the image detection frames respectively corresponding to the driver and other passengers in each passenger position fusion image" is refined into "determining the first frame center point of the driver's image detection frame in each passenger position fusion image, and the second frame center point of the image detection frame of other passengers; determining the first connection vector between the driver's position relative to the positions of other passengers in the corresponding passenger position fusion image, and the second connection vector between the positions of other passengers relative to the driver's position according to the first frame center point and the second frame center point; determining the influence degree vector between the driver and the other passengers based on the emotional states and emotional levels respectively corresponding to the driver and the other passengers according to the first connection vector and the second connection vector; determining the degree of emotional influence between the driver and other passengers in each passenger seat fusion image according to the influence degree vector." so as to improve the method for determining the degree of emotional influence between the driver and other passengers.

[0080] It should be noted that for the parts not described in detail in the embodiments of the present invention, reference can be made to the descriptions of other embodiments. Figure 5 As shown, the method comprises the following specific steps:

[0081] S310. When it is detected that there is an occupant boarding the target vehicle from outside, determine the external occupant's facial image and external occupant's height, and determine the internal occupant's facial image and internal occupant's height; the internal occupant's facial image includes at least one image detection frame of the internal occupant; the internal occupants include at least the driver.

[0082] S320. Based on the height of the external occupant and the height of the internal occupant, and the relative positions between the image detection frames of each internal occupant in the internal occupant facial image, perform image fusion on the external occupant facial image and the internal occupant facial image to determine occupant seat fusion images corresponding to at least one occupant seat arrangement combination; the occupant seat fusion images include the driver and other occupants; the other occupants are occupants other than the driver.

[0083] S330: Input the passenger seat fusion image into a pre-trained emotional state prediction model to obtain the emotional states and emotional levels of the driver and other passengers respectively output by the model.

[0084] S340, determining a first frame center point of the driver's image detection frame in the fusion image of each occupant's position, and a second frame center point of the image detection frame of other occupants.

[0085] The center point of the image detection frame may be the intersection point of the diagonal lines of the image detection frame. The center point of the driver's image detection frame is determined as the first center point; and the center points of the other passengers' image detection frames are determined as the second center points.

[0086] S350. Determine, based on the first frame center point and the second frame center point, a first connection vector between the driver's position relative to the positions of other occupants in the corresponding occupant position fusion image, and a second connection vector between the positions of other occupants relative to the driver's position.

[0087] According to the first coordinate point corresponding to the center point of the first frame and the second coordinate point corresponding to the center point of the second frame, based on the direction from the center point of the first frame to the center point of the second frame, a first connection vector is generated; according to the first coordinate point corresponding to the center point of the first frame and the second coordinate point corresponding to the center point of the second frame, based on the direction from the center point of the second frame to the center point of the first frame, a second connection vector is generated.

[0088] like Figure 6A schematic diagram of a vector expression of a first connection vector and a second connection vector is shown. The vector magnitude of the first connection vector between the driver and other occupant A is determined based on the first frame center point of the driver and the second frame center point of other occupant A; the vector direction of the first connection vector between the driver and other occupant A is the direction from the first frame center point of the driver to the second frame center point of other occupant A. The vector magnitude of the first connection vector between the driver and other occupant B is determined based on the first frame center point of the driver and the second frame center point of other occupant B. The vector magnitude of the first connection vector between the driver and other occupant B is determined based on the first frame center point of the driver and the second frame center point of other occupant B; the vector direction of the first connection vector between the driver and other occupant B is the direction from the first frame center point of the driver to the second frame center point of other occupant B.

[0089] The vector magnitude of the second connection vector between other occupant A and the driver is determined based on the second frame center point of other occupant A and the first frame center point of the driver; the vector direction of the second connection vector between other occupant A and the driver is the direction from the second frame center point of other occupant A to the first frame center point of the driver. The vector magnitude of the second connection vector between other occupant B and the driver is determined based on the second frame center point of other occupant B and the first frame center point of the driver; the vector direction of the second connection vector between other occupant B and the driver is the direction from the second frame center point of other occupant B to the first frame center point of the driver.

[0090] S360: Determine, according to the first connection vector and the second connection vector, an influence degree vector between the driver and the other occupants based on their respective emotional states and emotional levels.

[0091] Exemplarily, the emotion level is quantized into a unit length vector, specifically, emotion level 1 corresponds to 1 unit length vector; emotion level 2 corresponds to 2 unit length vectors; emotion level 3 corresponds to 3 unit length vectors; emotion level 4 corresponds to 4 unit length vectors; emotion level 5 corresponds to 5 unit length vectors, etc.

[0092] Among them, the influence degree vector between the driver and other passengers can represent the intensity of the emotional influence between the driver and other passengers.

[0093] In an optional embodiment, according to the first connection vector and the second connection vector, based on the emotional states and emotional levels of the driver and other passengers respectively, an influence degree vector between the driver and other passengers is determined, including: determining a first level vector corresponding to the driver according to the driver's emotional level; determining a second level vector corresponding to other passengers according to the emotional level of other passengers; determining a first influence degree vector between the driver and other passengers according to the first connection vector and the first level vector; determining a second influence degree vector between other passengers and the driver according to the second connection vector and the second level vector; and using the first influence degree vector and the second influence degree vector as the influence degree vector between the driver and other passengers.

[0094] like Figure 7 A schematic diagram of expressing an influence degree vector is shown. Figure 7 An example is given for describing the determination of the first influence degree vector between the driver and other passengers. The first level vector is a vector in the horizontal direction, and the first level vector is projected on the first connection vector, and the vector obtained after the projection is used as the first influence degree vector. The second influence degree vector between other passengers and the driver is determined in the same manner as the first influence degree vector, and this embodiment will not be described in detail.

[0095] S370. Determine the degree of emotional influence between the driver and other passengers in the fused image of each passenger seat according to the influence degree vector.

[0096] The influence degree vector includes a first influence degree vector between the driver and other passengers and a second influence degree vector between other passengers and the driver.

[0097] In an optional embodiment, the degree of emotional influence between the driver and other passengers in the fused image of each passenger seat is determined according to the influence degree vector, including: determining the first vector projection value and the second vector projection value of the first influence degree vector and the second influence degree vector in the same horizontal direction respectively; based on the emotional state of the driver and the emotional state of the other passengers, determining the degree of emotional influence between the driver and other passengers in the fused image of each passenger seat according to the first vector projection value and the second vector projection value.

[0098] The projection value of the first influence degree vector in the same horizontal direction or vertical direction is determined as the first vector projection value; and the projection value of the second influence degree vector in the same horizontal direction or vertical direction is determined as the second vector projection value. The first vector projection value and the second vector projection value may be vector projection lengths. Specifically, for the first influence degree vector, its angle with the horizontal direction is determined, and based on the angle and the vector length of the first influence degree vector, the first vector projection value may be determined. For example, Figure 8 A schematic diagram showing a method for determining a vector projection value is shown in FIG. If the vector length of the first influence degree vector is 6, and its unit can be pixel points, then the first vector projection value is The method for determining the second vector projection value is the same as the method for determining the first vector projection value, and this embodiment will not be described in detail.

[0099] According to the first vector projection value and the second vector projection value, based on the corresponding emotional states of the driver and other passengers, the degree of emotional interaction between the driver and other passengers is determined. Specifically, if the corresponding emotional state of the driver or other passengers is a positive state, such as happy or joyful, a positive value is assigned to the corresponding first vector projection value or second vector projection value; if the corresponding emotional state of the driver or other passengers is a negative state, such as angry or sad, a negative value is assigned to the corresponding first vector projection value or second vector projection value. For example, if the driver's emotional state is a happy state, its first vector projection value is a positive value, such as 6; if the driver's emotional state is a sad state, its first vector projection value is a negative value, such as -6.

[0100] For the driver and other passenger A, if the projection value of the first vector corresponding to the driver relative to other passenger A is 6, and the projection value of the second vector corresponding to other passenger A relative to the driver is -4, then the degree of emotional interaction between the driver and other passenger A is 2. Among them, a positive value indicates that the emotional interaction is a positive effect, a negative value indicates that the emotional interaction is a negative effect, and the numerical value indicates the degree.

[0101] S380. Determine the degree of emotional influence between the driver and other passengers in each passenger seat arrangement combination according to the degree of emotional influence between the driver and other passengers in the corresponding passenger seat fusion image.

[0102] S390: Determine a target passenger seat arrangement and combination according to the degree of emotional impact corresponding to each passenger seat arrangement and combination.

[0103] In a specific embodiment, in the passenger seat fusion image M, the emotional effect degree between the driver and the other passenger A is 2, the emotional effect degree between the driver and the other passenger B is -4, and the emotional effect degree between the driver and the other passenger C is 3, then the driver of the passenger seat fusion image M has a total emotional effect degree between the other passengers of 1. In the passenger seat fusion image N, the emotional effect degree between the driver and the other passenger A is -2, the emotional effect degree between the driver and the other passenger B is 3, and the emotional effect degree between the driver and the other passenger C is -4, then the driver of the passenger seat fusion image N has a total emotional effect degree between the other passengers of -3. In the passenger seat fusion image Q, the emotional effect degree between the driver and the other passenger A is 2, the emotional effect degree between the driver and the other passenger B is 5, and the emotional effect degree between the driver and the other passenger C is -3, then the driver of the passenger seat fusion image Q has a total emotional effect degree between the other passengers of 4. Then the passenger seat arrangement and combination mode corresponding to the passenger seat fusion image Q is used as the target passenger seat arrangement and combination mode.

[0104] The technical solution of this embodiment determines the first frame center point of the driver's image detection frame in each passenger position fusion image, and the second frame center point of the image detection frame of other passengers; according to the first frame center point and the second frame center point, determines the first connection vector between the driver's position relative to the position of other passengers in the corresponding passenger position fusion image, and the second connection vector between the position of other passengers relative to the position of the driver; according to the first connection vector and the second connection vector, based on the emotional state and emotional level corresponding to the driver and other passengers, determines the influence degree vector between the driver and the other passengers; according to the influence degree vector, determines the emotional influence degree between the driver and other passengers in each passenger seat fusion image. The above technical solution characterizes the emotional influence degree between the driver and other passengers by generating a vector, and in the determination process, considers the influence of the emotional level and emotional state, as well as the influence of the position distance between the driver and other passengers, thereby improving the accuracy of determining the emotional influence degree, and further improving the accuracy of determining the emotional influence degree.

[0105] Embodiment 4

[0106] Fig. 9 The fourth embodiment of the present invention provides a structural diagram of an occupant seat arrangement device based on emotional state. The embodiment of the present invention provides an occupant seat arrangement device based on emotional state, which can be used to reasonably allocate occupant seats to occupants of a target vehicle to reduce the impact on the driver's emotions. The occupant seat arrangement device based on emotional state can be implemented in the form of hardware and / or software, such as Fig. 9As shown, the occupant seat arrangement device based on emotional state can be configured in an electronic device, and specifically includes: an occupant image determination module 401, an image fusion module 402, an image analysis module 403 and a seat arrangement determination module 404.

[0107] The occupant image determination module 401 is used to determine the external occupant's facial image and the external occupant's height when detecting that there is an occupant boarding action outside the target vehicle, and determine the internal occupant's facial image and the internal occupant's height of the internal occupant of the target vehicle; the internal occupant's facial image includes at least one image detection frame of the internal occupant; the internal occupant includes at least the driver;

[0108] The image fusion module 402 is used to fuse the external occupant facial image and the internal occupant facial image according to the external occupant height and the internal occupant height, and the relative positions between the image detection frames of each internal occupant in the internal occupant facial image, and determine the occupant seat fusion images corresponding to at least one occupant seat arrangement combination; the occupant seat fusion images include the driver and other occupants; the other occupants are occupants other than the driver;

[0109] An image analysis module 403 is used to perform image analysis on the passenger seat fusion image to determine the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination;

[0110] The seat arrangement determination module 404 is used to determine a target passenger seat arrangement combination according to the emotion impact degree corresponding to each passenger seat arrangement combination.

[0111] The above technical solution combines the height and relative position of the passengers for image fusion and analyzes the emotional state of the passengers, thereby realizing the degree of emotional influence between the driver and other passengers under different passenger seat arrangements. The optimal passenger seat arrangement is selected based on the degree of emotional influence, so that the influence of other passengers on the driver's emotions is minimized, thereby improving vehicle driving safety and passenger riding comfort.

[0112] Optionally, the image analysis module 403 includes:

[0113] An emotional state determination unit, used for inputting the passenger seat fusion image into a pre-trained emotional state prediction model to obtain the emotional states and emotional levels of the driver and other passengers respectively output by the model;

[0114] an emotion effect degree determination unit, for determining the emotion effect degree between the driver and other passengers in each of the passenger seat fused images, according to the emotion states and emotion levels respectively corresponding to the driver and other passengers, and the detection frame positions of the image detection frames respectively corresponding to the driver and other passengers in each of the passenger position fused images;

[0115] The emotion influence degree determination unit is used to determine the emotion influence degree between the driver and other passengers in each passenger seat arrangement combination according to the emotion influence degree between the driver and other passengers in the corresponding passenger seat fusion image.

[0116] Optionally, the emotion effect degree determination unit comprises:

[0117] A center point determination subunit, used to determine a first frame center point of the driver's image detection frame in each of the occupant position fusion images, and a second frame center point of the image detection frames of other occupants;

[0118] a connection vector determination subunit, for determining, according to the first frame center point and the second frame center point, a first connection vector between the position of the driver relative to the positions of other occupants in the corresponding occupant position fusion image, and a second connection vector between the positions of other occupants relative to the position of the driver;

[0119] an influence degree vector determining subunit, configured to determine an influence degree vector between the driver and the other occupants based on the first connection vector and the second connection vector and on the emotional states and emotional levels respectively corresponding to the driver and the other occupants;

[0120] The emotion effect degree determination subunit is used to determine the emotion effect degrees between the driver and other passengers in each of the passenger seat fusion images according to the influence degree vector.

[0121] Optionally, the influence degree vector determination subunit is specifically used to:

[0122] Determining a first level vector corresponding to the driver according to the driver's emotion level;

[0123] Determining, according to the emotion levels of the other passengers, a second level vector corresponding to the other passengers;

[0124] determining a first influence degree vector between the driver and the other occupants according to the first connection vector and the first level vector;

[0125] determining a second influence degree vector between the other occupants and the driver according to the second connection vector and the second level vector;

[0126] The first influence degree vector and the second influence degree vector are used as influence degree vectors between the driver and the other occupants.

[0127] Optionally, the emotion effect degree determination subunit is specifically used for:

[0128] Determine a first vector projection value and a second vector projection value of the first influence degree vector and the second influence degree vector respectively in the same horizontal direction;

[0129] According to the first vector projection value and the second vector projection value, based on the emotional state of the driver and the emotional states of the other passengers, the degree of emotional interaction between the driver and the other passengers in each of the passenger seat fusion images is determined.

[0130] Optionally, the model training method of the emotional state prediction model is as follows:

[0131] Obtain portrait images of people in different emotional states as sample data sets;

[0132] Performing sample labeling on the portrait images in the sample data set to obtain sample labels of the standard emotional state and standard emotional level corresponding to each portrait image;

[0133] Input the sample data set with sample labels into the pre-built network model to obtain the predicted emotional state and predicted emotional level output by the model;

[0134] According to the standard emotional state, standard emotional level, predicted emotional state and predicted emotional level, the network model is trained until a preset model training end condition is met to obtain an emotional state prediction model.

[0135] The passenger seat arrangement device based on emotional state provided in the embodiment of the present invention can execute the passenger seat arrangement method based on emotional state provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0136] Embodiment 5

[0137] Fig.10A schematic diagram of an electronic device 50 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0138] like Fig.10 As shown, the electronic device 50 includes at least one processor 51, and a memory connected to the at least one processor 51 in communication, such as a read-only memory (ROM) 52, a random access memory (RAM) 53, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 51 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 52 or the computer program loaded from the storage unit 58 to the random access memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 50 can also be stored. The processor 51, the ROM 52, and the RAM 53 are connected to each other through a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0139] A number of components in the electronic device 50 are connected to the I / O interface 55, including: an input unit 56, such as a keyboard, a mouse, etc.; an output unit 57, such as various types of displays, speakers, etc.; a storage unit 58, such as a disk, an optical disk, etc.; and a communication unit 59, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0140] The processor 51 may be a variety of general and / or dedicated processing components with processing and computing capabilities. Some examples of the processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 51 executes the various methods and processes described above, such as the passenger seat arrangement method based on emotional state.

[0141] In some embodiments, the method for arranging passenger seats based on emotional states may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 50 via the ROM 52 and / or the communication unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the method for arranging passenger seats based on emotional states described above may be performed. Alternatively, in other embodiments, the processor 51 may be configured to execute the method for arranging passenger seats based on emotional states in any other appropriate manner (e.g., by means of firmware).

[0142] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0144] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0147] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0148] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0149] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for arranging passenger seats based on emotional state, characterized in that: include: When an action of a passenger boarding the target vehicle is detected outside, an external passenger facial image and an external passenger height of the external passenger are determined, and an internal passenger facial image and an internal passenger height of an internal passenger of the target vehicle are determined; the internal passenger facial image includes at least one image detection frame of the internal passenger; the internal passenger includes at least the driver; According to the height of the external occupant and the height of the internal occupant, and the relative positions between the image detection frames of each of the internal occupants in the internal occupant facial image, the external occupant facial image and the internal occupant facial image are image-fused to determine occupant seat fusion images corresponding to at least one occupant seat arrangement and combination mode; the occupant seat fusion image includes the driver and other occupants; the other occupants are occupants other than the driver; Inputting the passenger seat fusion image into a pre-trained emotional state prediction model to obtain the emotional states and emotional levels of the driver and other passengers respectively output by the model; Determining the degree of emotional interaction between the driver and the other passengers in each of the passenger seat fused images according to the emotional states and emotional levels respectively corresponding to the driver and the other passengers, and the detection frame positions of the image detection frames respectively corresponding to the driver and the other passengers in each of the passenger seat fused images; Determining the degree of emotional influence between the driver and other passengers in each of the passenger seat arrangement combinations according to the degree of emotional influence between the driver and other passengers in the corresponding passenger seat fusion images; A target passenger seat arrangement and combination is determined according to the emotion impact degrees corresponding to each of the passenger seat arrangement and combination modes.

2. The method according to claim 1, characterized in that The determining the degree of emotional interaction between the driver and the other passengers in each of the passenger seat fused images according to the emotional states and emotional levels respectively corresponding to the driver and the other passengers, and the detection frame positions of the image detection frames respectively corresponding to the driver and the other passengers in each of the passenger seat fused images, comprises: Determine a first frame center point of the driver's image detection frame in each of the passenger seat fusion images, and a second frame center point of the image detection frames of other passengers; Determine, according to the first frame center point and the second frame center point, a first connection vector between the position of the driver relative to the positions of other passengers in the corresponding passenger seat fusion image, and a second connection vector between the positions of other passengers relative to the position of the driver; Determining, according to the first connection vector and the second connection vector, an influence degree vector between the driver and the other occupants based on the emotional states and emotional levels respectively corresponding to the driver and the other occupants; The degree of emotional interaction between the driver and other passengers in each of the passenger seat fusion images is determined according to the influence degree vector.

3. The method according to claim 2, characterized in that The determining, according to the first connection vector and the second connection vector, the influence degree vector between the driver and the other occupants based on the emotional states and emotional levels respectively corresponding to the driver and the other occupants, comprises: Determining a first level vector corresponding to the driver according to the driver's emotion level; Determining, according to the emotion levels of the other passengers, a second level vector corresponding to the other passengers; determining a first influence degree vector between the driver and the other occupants according to the first connection vector and the first level vector; determining a second influence degree vector between the other occupants and the driver according to the second connection vector and the second level vector; The first influence degree vector and the second influence degree vector are used as influence degree vectors between the driver and the other occupants.

4. The method according to claim 3, characterized in that: Determining the degree of emotional interaction between the driver and other passengers in each of the passenger seat fusion images according to the influence degree vector includes: Determine a first vector projection value and a second vector projection value of the first influence degree vector and the second influence degree vector respectively in the same horizontal direction; According to the first vector projection value and the second vector projection value, based on the emotional state of the driver and the emotional states of the other passengers, the degree of emotional interaction between the driver and the other passengers in each of the passenger seat fusion images is determined.

5. The method according to claim 1, characterized in that The model training method of the emotional state prediction model is as follows: Obtain portrait images of people in different emotional states as sample data sets; Performing sample labeling on the portrait images in the sample data set to obtain sample labels of the standard emotional state and standard emotional level corresponding to each portrait image; Input the sample data set with sample labels into the pre-built network model to obtain the predicted emotional state and predicted emotional level output by the model; According to the standard emotional state, standard emotional level, predicted emotional state and predicted emotional level, the network model is trained until a preset model training end condition is met to obtain an emotional state prediction model.

6. An occupant seat arrangement device based on emotional state, characterized in that: include: An occupant image determination module is used to determine an external occupant's facial image and an external occupant's height when detecting an occupant's boarding action outside the target vehicle, and to determine an internal occupant's facial image and an internal occupant's height inside the target vehicle; the internal occupant's facial image includes at least one image detection frame of the internal occupant; the internal occupant includes at least the driver; an image fusion module, configured to fuse the external occupant facial image and the internal occupant facial image according to the external occupant height and the internal occupant height, and the relative positions between the image detection frames of each of the internal occupants in the internal occupant facial image, and determine occupant seat fusion images corresponding to at least one occupant seat arrangement and combination; the occupant seat fusion images include the driver and other occupants; the other occupants are occupants other than the driver; An image analysis module, used for performing image analysis on the passenger seat fusion image to determine the degree of emotional influence between the driver and other passengers under each passenger seat arrangement combination; A seat arrangement determination module, used to determine a target passenger seat arrangement combination according to the emotional impact degree corresponding to each passenger seat arrangement combination; The image analysis module comprises: An emotional state determination unit, used for inputting the passenger seat fusion image into a pre-trained emotional state prediction model to obtain the emotional states and emotional levels of the driver and other passengers respectively output by the model; an emotion effect degree determination unit, for determining the emotion effect degree between the driver and the other passengers in each of the passenger seat fused images, according to the emotion states and emotion levels respectively corresponding to the driver and the other passengers, and the detection frame positions of the image detection frames respectively corresponding to the driver and the other passengers in each of the passenger seat fused images; The emotion influence degree determination unit is used to determine the emotion influence degree between the driver and other passengers in each passenger seat arrangement combination according to the emotion influence degree between the driver and other passengers in the corresponding passenger seat fusion image.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method for arranging passenger seats based on emotional states according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for arranging passenger seats based on emotional states according to any one of claims 1 to 5 when executed.

9. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the method for arranging passenger seats based on emotional states according to any one of claims 1 to 5.

Images