Method, device, computer equipment and storage medium for identifying relationship between people

Abstract

The present application relates to a method, device, computer equipment and storage medium for character relationship recognition. The method comprises: collecting a target vehicle interior picture, wherein the target vehicle interior picture contains a first person and a second person, and records the facial information and body information of the first person and the second person; obtaining the facial coordinates of the first person and the second person through a face detector; and processing the target vehicle interior picture, the facial coordinates of the first person and the second person using a character relationship recognition model to obtain a character relationship recognition result between the first person and the second person. The embodiments of the present application can improve the recognition accuracy of the character relationship in the vehicle.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, apparatus, computer device and storage medium for identifying person relationships. Background Art

[0002] It is difficult to judge the relationship between the driver and the passengers, and it is impossible to detect danger in time, and there is a lack of effective means. Although most online ride-hailing vehicles are currently equipped with monitoring equipment, these monitoring devices only have the function of shooting, and specific behavior identification needs to be completed manually afterwards, and it is impossible to detect and stop it in time while the vehicle is driving. Due to practical conditions, it is impossible to equip each vehicle with manpower to monitor the entire process to provide early warning of dangerous behavior.

[0003] At present, some solutions have proposed to use deep learning methods to judge the emotions of drivers and passengers. For example, the target detection network is used to detect the position of the face, and the face image is passed to the emotion detection network to judge the emotions of the person. When the excited emotions are identified, an early warning is issued, or manual intervention is performed. However, some people's emotions are not reflected on their faces, and their emotions are not expressed on their faces, so it is difficult to judge their true emotions. In this case, the accuracy of the above solutions is not high. Summary of the invention

[0004] In view of the above-mentioned deficiencies or shortcomings, the present application provides a method, apparatus, computer device and storage medium for identifying character relationships. The embodiments of the present application can improve the accuracy of identifying character relationships in a vehicle.

[0005] The present application provides a method for identifying character relationships according to a first aspect. In one embodiment, the method includes:

[0006] Collecting a target vehicle interior picture, wherein the target vehicle interior picture includes a first person and a second person, and records facial information and body information of the first person and the second person;

[0007] Acquire the face coordinates of the first person and the face coordinates of the second person through a face detector;

[0008] The target vehicle interior image, the facial coordinates of the first person, and the facial coordinates of the second person are processed using a person relationship recognition model to obtain a person relationship recognition result between the first person and the second person.

[0009] In one embodiment, the character relationship recognition result is used to indicate whether there is a contradiction between the first character and the second character;

[0010] When the character relationship recognition result indicates that there is a contradiction between the first character and the second character, the method further includes:

[0011] Collect multiple temporally continuous images of the target vehicle interior;

[0012] Detect whether the relationships between the first person and the second person in each target vehicle interior image are contradictory;

[0013] If so, notify the human to coordinate the character relationship between the first character and the second character.

[0014] In one embodiment, the training process of the character relationship recognition model includes:

[0015] Constructing a training sample set, the training sample set includes multiple training samples, and the training samples are pictures inside a car including a first person and a second person;

[0016] Label each training sample, and the label used for labeling is used to indicate whether there is any contradiction between the first character and the second character;

[0017] Label the face frames of the first person and the second person in each training sample;

[0018] Based on a preset deep learning training framework, model training is performed according to the multiple training samples and the labeled data corresponding to each training sample to obtain a trained character relationship recognition model.

[0019] In one embodiment, the method is applied to a camera device having a computing function;

[0020] The method also includes: deploying the trained character relationship recognition model on a camera device.

[0021] In one embodiment, a character relationship recognition model is used to process the target vehicle interior image, the facial coordinates of the first character, and the facial coordinates of the second character to obtain a character relationship recognition result between the first character and the second character, including:

[0022] Assume the pixel size of the target car interior image is M*N;

[0023] The feature extraction network is used to extract features from the target vehicle interior image to obtain a first feature map with a pixel size of M1*N1;

[0024] Convert the facial coordinates of the first person and the facial coordinates of the second person into the first feature map in proportion, and extract two second feature maps with area sizes of Wd*Hd and Wp*Hp from the first feature map;

[0025] Using a feature extraction network to extract features from the two second feature maps respectively, to obtain two third feature maps;

[0026] The two third feature maps are adjusted to the same pixel size using linear interpolation, and feature vectors of dimension V are extracted from the two third feature maps after the pixel size adjustment using a fully connected network;

[0027] Perform matrix calculation on the two extracted feature vectors to obtain a weight matrix with a dimension of V*V for characterizing whether the facial emotions between the first person and the second person are similar, and interpolate the weight matrix to obtain a weight feature map with a pixel size of M1*N1;

[0028] Add the weighted feature map to the first feature map to get a new first feature map with a pixel size of M1*N1

[0029] The feature extraction network is used to extract features from the new first feature map, and the sigmoid function is used to classify the features extracted from the new first feature map to obtain a character relationship recognition result between the first character and the second character.

[0030] In one embodiment, before collecting the target vehicle interior image, the method further includes:

[0031] Collect the sound inside the car;

[0032] Detect whether the collected sound contains human voice;

[0033] Collect pictures of the target vehicle, including:

[0034] In response to the collected sound containing a human voice, a picture of the interior of the target vehicle is collected.

[0035] In one embodiment, collecting a target vehicle interior image includes:

[0036] Continuously take pictures inside the car;

[0037] Filter out the target in-car pictures from the captured in-car pictures.

[0038] According to a second aspect, the present application provides a device for identifying a person relationship. In one embodiment, the device includes:

[0039] A collection module is used to collect a target vehicle interior picture, wherein the target vehicle interior picture includes a first person and a second person, and records facial information and body information of the first person and the second person;

[0040] A face coordinate acquisition module, used to acquire the face coordinates of the first person and the face coordinates of the second person through a face detector;

[0041] The character relationship recognition module is used to use the character relationship recognition model to process the target vehicle interior picture, the facial coordinates of the first character and the facial coordinates of the second character to obtain a character relationship recognition result between the first character and the second character.

[0042] According to a third aspect, the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above method embodiments when executing the computer program.

[0043] According to a fourth aspect, the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of an embodiment of any of the above methods are implemented.

[0044] In an embodiment of the present application, a target vehicle interior picture is collected, the target vehicle interior picture includes a first person and a second person, and the facial information and body information of the first person and the second person are recorded; the facial coordinates of the first person and the second person are obtained by a face detector; the target vehicle interior picture, the facial coordinates of the first person and the second person are processed using a character relationship recognition model to obtain a character relationship recognition result between the first person and the second person. When recognizing the relationship between the first person and the second person, the embodiment of the present application not only takes into account the facial information of the person, but also combines the body information of the person, thereby improving the recognition accuracy of the relationship between the people in the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of a process of identifying a person relationship in an embodiment;

[0046] Figure 2 A schematic diagram of a process of processing data by a person relationship recognition model in one embodiment;

[0047] Figure 3 A schematic diagram of the structure of a character relationship recognition model in one embodiment;

[0048] Figure 4 is a structural block diagram of a device for identifying a relationship between people in one embodiment;

[0049] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] The present application provides a method for identifying a relationship between people. In one embodiment, the method for identifying a relationship between people includes: Figure 2 Steps shown.

[0052] S110: Collecting a target vehicle interior image, where the target vehicle interior image includes a first person and a second person, and records facial information and body information of the first person and the second person;

[0053] The target in-car picture refers to an in-car picture that includes the first person and the second person and records the facial information and body information of the first person and the second person.

[0054] The first person and the second person refer to the driver and the passenger in the car. The target in-car image can be collected by a car camera. Since the installation position of the car camera in the car is usually fixed, the positions of the driver and the passenger are also fixed, for example, the driver is in the driver's seat and the passenger is in a non-driver's seat, such as the front passenger seat or the back seat. Therefore, the person in the driver's seat in the target in-car image can be determined as the first person (i.e., the driver), and the person in the non-driver's seat in the target in-car image can be determined as the second person (i.e., the passenger).

[0055] Facial information may refer to the face of a person, and body information may refer to the hands of a person.

[0056] S120: Acquire the face coordinates of the first person and the face coordinates of the second person through a face detector.

[0057] The face coordinates refer to the coordinates of the first person or the second person in the target vehicle interior image.

[0058] S130: Using a character relationship recognition model, the target vehicle interior image, the facial coordinates of the first character, and the facial coordinates of the second character are processed to obtain a character relationship recognition result between the first character and the second character.

[0059] The character relationship recognition result is used to characterize whether there is a contradiction between the first character and the second character.

[0060] In this embodiment, it is considered that when a conflict occurs between the driver and the passenger, both will be emotionally excited, angry and have physical conflict, while when the two are emotionally stable or have different emotions (e.g., one is emotionally stable and the other is emotionally excited), there is usually no conflict. Therefore, when identifying the relationship between people in the car, this embodiment not only considers the facial emotion information of the faces of the people in the car, but also comprehensively considers the body movements of the first person and the second person, that is, the driver and the passenger. Compared with the existing method of simply relying on facial features to perform emotion recognition and character relationship recognition, it can more accurately judge whether the driver and the passenger are in an excited state, physical conflict and other extreme behaviors, which is conducive to resolving the conflict before the conflict escalates through early warning or manual intervention.

[0061] In one embodiment, the training process of the above-mentioned character relationship recognition model includes:

[0062] Constructing a training sample set, the training sample set includes multiple training samples, and the training samples are pictures inside a car including a first person and a second person;

[0063] Each training sample is labeled, and the labeled label is used to characterize whether there is any contradiction between the first character and the second character; for example, label 0 indicates no contradiction (i.e., the emotions of the first character and the second character are stable or different), and label 1 indicates contradiction (i.e., the first character and the second character are both emotionally excited).

[0064] Label the face frames of the first person and the second person in each training sample;

[0065] Based on a preset deep learning training framework, model training is performed according to the multiple training samples and the labeled data corresponding to each training sample to obtain a trained character relationship recognition model.

[0066] Among them, the preset deep learning training framework can be a conventional deep learning training framework, and this embodiment does not specifically limit which training framework is adopted.

[0067] Furthermore, the trained model can be deployed on a vehicle-mounted device or on a server. In one embodiment, the method for identifying relationships between people provided in this embodiment can be applied to a camera device with a computing function; accordingly, the method further includes: deploying the trained model for identifying relationships between people on the camera device. In this way, the camera device can be used for real-time tracking and monitoring without the need for human supervision, thus reducing human costs.

[0068] In one embodiment, a character relationship recognition model is used to process the target vehicle interior image, the facial coordinates of the first character, and the facial coordinates of the second character to obtain a character relationship recognition result between the first character and the second character, such as Figure 2 As shown, including:

[0069] S131: Assuming that the pixel size of the target vehicle interior image is M*N, a feature extraction network is used to extract features of the target vehicle interior image to obtain a first feature map with a pixel size of M1*N1;

[0070] S132: Convert the facial coordinates of the first person and the facial coordinates of the second person into the first feature map in proportion, and extract two second feature maps with area sizes of Wd*Hd and Wp*Hp from the first feature map;

[0071] S133: using a feature extraction network to perform feature extraction on the two second feature maps respectively to obtain two third feature maps;

[0072] S134: using linear interpolation to adjust the two third feature maps to the same pixel size, and using a fully connected network to extract feature vectors with a dimension of V from the two third feature maps after the pixel sizes are adjusted;

[0073] S135: performing matrix calculation on the two extracted feature vectors to obtain a weight matrix with a dimension of V*V for characterizing whether the facial emotions of the first person and the second person are similar, and interpolating the weight matrix to obtain a weight feature map with a pixel size of M1*N1;

[0074] S136: Add the weighted feature map to the first feature map to obtain a new first feature map with a pixel size of M1*N1;

[0075] S137: Extract features from the new first feature map using a feature extraction network, and classify the features extracted from the new first feature map using a sigmoid function to obtain a character relationship recognition result between the first character and the second character.

[0076] In this embodiment, the input of the model (referring to the person relationship recognition model) is the target vehicle interior picture, the face coordinates of the first person and the face coordinates of the second person.

[0077] The following is an example to illustrate the model processing process.

[0078] The structure of the model in this example is shown in Figure 3 , the model processing process is as follows:

[0079] It should be noted that the model needs to extract image features, so a feature extraction network (such as resnet) is used as the basic module to extract features. First, the face detector is used to obtain the face coordinates of the driver and the passenger, and the target car interior image is used as the model input. Assume that the pixel size of the input image is M*N;

[0080] Use resnet to extract features from the image with a pixel size of M*N, and obtain a feature map with a pixel size of M1xN1, i.e., the first feature map;

[0081] The obtained facial coordinates of the driver and the passenger are converted into the first feature map in proportion, and two feature maps with area sizes of Wd*Hd and Wp*Hp are extracted from the first feature map, namely the second feature map; the features of these two parts are extracted separately using resnet to obtain two feature maps, namely the third feature map; the two third feature maps are adjusted to the same pixel size by linear interpolation, and the feature vectors of the two third feature maps are extracted using a fully connected network, with a dimension of V;

[0082] Perform matrix calculation on the two eigenvectors to obtain a correlation matrix with a dimension of V*V, which is used as a weight matrix for determining whether the facial emotions are similar. Then, a weight feature map with a pixel size of M1*N1 is obtained by interpolation.

[0083] Add the interpolated weighted feature map to the first feature map of original size M1*N1 to obtain a new first feature map of M1*N1;

[0084] Resnet is used to extract features from the new first feature map, and the sigmoid function is used for classification to determine whether there is any contradiction or conflict between the driver and the passenger.

[0085] In this example, facial emotion information is obtained by extracting features from the driver and passenger's faces separately, and a similarity matrix (i.e., weight matrix) is established to establish weights for whether the driver and passenger have the same emotion. The weighted feature map can reflect scene and physical conflict information. The similarity of facial emotions is used to comprehensively judge whether there is a driver-passenger conflict. The driver-passenger conflict judgment result can be directly obtained by simply inputting the driver and passenger pictures and facial information.

[0086] In one embodiment, when the character relationship recognition result indicates that there is a contradiction between the first character and the second character, the method further includes:

[0087] Collect multiple temporally continuous images of the target vehicle interior;

[0088] Detect whether the relationships between the first person and the second person in each target vehicle interior image are contradictory;

[0089] If so, notify the human to coordinate the character relationship between the first character and the second character.

[0090] In this embodiment, in order to improve the accuracy, the sliding judgment can be performed through multiple frames. If the driver-passenger conflict is detected in a certain number of consecutive frames, a warning can be issued and recorded, and manual intervention can be performed in advance.

[0091] In one embodiment, before collecting the target vehicle interior image, the method further includes: collecting vehicle interior sound; detecting whether the collected sound contains a human voice;

[0092] Accordingly, collect pictures inside the target vehicle, including:

[0093] In response to the collected sound containing a human voice, a picture of the interior of the target vehicle is collected.

[0094] In this embodiment, if there is no voice, the possibility of a conflict between the first person and the second person is extremely low. Therefore, collecting pictures after recognizing the voice in the car can effectively reduce the workload.

[0095] In one embodiment, collecting the target vehicle interior picture includes: continuously taking vehicle interior pictures; and filtering out the target vehicle interior picture from the taken vehicle interior pictures.

[0096] In this embodiment, the person in the car may perform some activities (such as turning the head, lowering the head, etc.), which may result in the captured interior photo not containing the complete facial information and / or body information of the person (referring to the first person and / or the second person). Therefore, when it is necessary to collect the target interior photo, multiple photos are taken continuously through a photographing device such as a car-mounted camera, and then one (or one frame) of the interior photo is selected as the target interior photo. This can improve the recognition accuracy of the relationship between the characters.

[0097] Figure 1-2 FIG. 1 is a flow chart of a method for identifying a person relationship in one embodiment. It should be understood that although Figure 1-2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0098] Based on the same inventive concept, the present application also provides a person relationship recognition device. Figure 4 As shown, the character relationship recognition device includes the following modules:

[0099] The acquisition module 110 is used to acquire a target vehicle interior picture, wherein the target vehicle interior picture includes a first person and a second person, and records facial information and body information of the first person and the second person;

[0100] A face coordinate acquisition module 120, configured to acquire the face coordinates of the first person and the face coordinates of the second person through a face detector;

[0101] The character relationship recognition module 130 is used to use the character relationship recognition model to process the target vehicle interior picture, the facial coordinates of the first character and the facial coordinates of the second character to obtain a character relationship recognition result between the first character and the second character.

[0102] In one embodiment, the character relationship recognition result is used to indicate whether there is any contradiction between the first character and the second character.

[0103] The acquisition module is further used to acquire a plurality of temporally continuous images of the interior of the target vehicle when the character relationship recognition result indicates that there is a conflict between the first character and the second character;

[0104] The device also includes:

[0105] A detection module, used to detect whether the relationships between the first person and the second person in each target vehicle interior picture are contradictory;

[0106] The notification module is used to notify a person to coordinate the relationship between the first person and the second person when there is a conflict between the relationship between the first person and the second person in each target vehicle interior picture.

[0107] In one embodiment, the apparatus further comprises:

[0108] A training sample set construction module is used to construct a training sample set, where the training sample set includes a plurality of training samples, and the training samples are pictures inside a car including a first person and a second person;

[0109] A first labeling module is used to label each training sample, and the label used for labeling is used to indicate whether there is any contradiction between the first character and the second character;

[0110] The second annotation module is used to annotate the face frames of the first person and the second person in each training sample;

[0111] The model training module is used to perform model training based on the preset deep learning training framework according to the multiple training samples and the labeled data corresponding to each training sample to obtain a trained character relationship recognition model.

[0112] In one embodiment, the apparatus is a photographing device with computing functions; the apparatus also includes a deployment module, which is used to deploy the trained character relationship recognition model on the photographing device.

[0113] In one embodiment, the character relationship identification module includes:

[0114] A first feature map acquisition submodule is used to set the pixel size of the target vehicle interior image to M*N, use the feature extraction network to extract features of the target vehicle interior image, and obtain a first feature map with a pixel size of M1*N1;

[0115] The second feature map acquisition submodule is used to convert the face coordinates of the first person and the face coordinates of the second person into the first feature map in proportion, and extract two second feature maps with area sizes of Wd*Hd and Wp*Hp from the first feature map;

[0116] A third feature map acquisition submodule is used to use a feature extraction network to extract features from the two second feature maps respectively to obtain two third feature maps;

[0117] A feature vector extraction submodule, used to adjust the two third feature maps to the same pixel size using linear interpolation, and to extract feature vectors of dimension V from the two third feature maps after the pixel size adjustment using a fully connected network;

[0118] A weight feature map acquisition submodule is used to perform matrix calculation on the two extracted feature vectors to obtain a weight matrix with a dimension of V*V for characterizing whether the facial emotions between the first person and the second person are similar, and to interpolate the weight matrix to obtain a weight feature map with a pixel size of M1*N1;

[0119] The new first feature map acquisition submodule is used to add the weight feature map to the first feature map to obtain a new first feature map with a pixel size of M1*N1

[0120] The classification submodule is used to extract features from the new first feature map using a feature extraction network, and to classify the features extracted from the new first feature map using a sigmoid function to obtain a character relationship recognition result between the first character and the second character.

[0121] In one embodiment, the device further includes a sound collection module and a human voice detection module.

[0122] A sound collection module is used to collect the sound inside the vehicle before collecting the image inside the target vehicle;

[0123] A human voice detection module is used to detect whether the collected sound contains human voice;

[0124] Correspondingly, the acquisition module is specifically configured to acquire a picture of the interior of the target vehicle in response to the acquired sound containing a human voice.

[0125] In one embodiment, the acquisition module is further configured to continuously capture in-vehicle images, and select target in-vehicle images from the captured in-vehicle images.

[0126] For the specific limitations of the character relationship recognition device, please refer to the limitations of the character relationship recognition method above, which will not be repeated here. The various modules in the above-mentioned character relationship recognition device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0127] In one embodiment, a computer device is provided, whose internal structure diagram can be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as pictures in the car. The specific stored data can also refer to the definition in the above method embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying character relationships is implemented.

[0128] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0129] This embodiment also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in any of the above method embodiments when executing the computer program.

[0130] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0131] Those skilled in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0132] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0133] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying character relationships, characterized in that: The method comprises: Collecting a target vehicle interior picture, wherein the target vehicle interior picture includes a first person and a second person, and records facial information and body information of the first person and the second person; Acquire the face coordinates of the first person and the face coordinates of the second person through a face detector; Using a character relationship recognition model, processing the target vehicle interior image, the facial coordinates of the first character, and the facial coordinates of the second character to obtain a character relationship recognition result between the first character and the second character; The using of the character relationship recognition model to process the target vehicle interior image, the facial coordinates of the first character, and the facial coordinates of the second character to obtain a character relationship recognition result between the first character and the second character includes: Assume that the pixel size of the target interior image is M*N; Using a feature extraction network to extract features from the target in-vehicle image, obtaining a first feature map with a pixel size of M1*N1; Convert the facial coordinates of the first person and the facial coordinates of the second person into the first feature map in proportion, and extract two second feature maps with area sizes of Wd*Hd and Wp*Hp from the first feature map; Using the feature extraction network to extract features from the two second feature maps respectively, to obtain two third feature maps; Using linear interpolation to adjust the two third feature maps to the same pixel size, and using a fully connected network to extract feature vectors with a dimension of V from the two third feature maps after the pixel size adjustment; Performing matrix calculation on the two extracted feature vectors to obtain a weight matrix with a dimension of V*V for characterizing whether the facial emotions of the first person and the second person are similar, and interpolating the weight matrix to obtain a weight feature map with a pixel size of M1*N1; Adding the weighted feature map to the first feature map to obtain a new first feature map with a pixel size of M1*N1; The feature extraction network is used to extract features from the new first feature map, and the features extracted from the new first feature map are classified using a sigmoid function to obtain a character relationship recognition result between the first character and the second character.

2. The method according to claim 1, characterized in that The character relationship recognition result is used to indicate whether there is any contradiction between the first character and the second character; When the character relationship recognition result indicates that there is a conflict between the first character and the second character, the method further includes: Collect multiple temporally continuous images of the target vehicle interior; Detecting whether the relationships between the first person and the second person in each target vehicle interior picture are contradictory; If so, notify someone to coordinate the character relationship between the first character and the second character.

3. The method according to claim 1, characterized in that The training process of the character relationship recognition model includes: Constructing a training sample set, wherein the training sample set includes a plurality of training samples, and the training samples are pictures inside a car including a first person and a second person; Labeling each of the training samples, wherein the label used for labeling is used to indicate whether there is any contradiction in the character relationship between the first character and the second character; Labeling the face frames of the first person and the second person in each of the training samples; Based on a preset deep learning training framework, model training is performed according to the multiple training samples and the labeled data corresponding to each training sample to obtain a trained character relationship recognition model.

4. The method according to claim 3, characterized in that The method is applied to a photographing device having a computing function; The method also includes: deploying the trained character relationship recognition model on the camera device.

5. The method according to claim 1, characterized in that The acquisition of the target vehicle interior image also includes: Collect the sound inside the car; Detect whether the collected sound contains human voice; The collecting of the target vehicle interior picture includes: In response to the collected sound containing a human voice, a picture of the interior of the target vehicle is collected.

6. The method according to claim 1, characterized in that The collecting of the target vehicle interior picture includes: Continuously take pictures inside the car; Filter out the target in-car pictures from the captured in-car pictures.

7. A person relationship recognition device, characterized in that: The device comprises: A collection module, used for collecting a target vehicle interior picture, wherein the target vehicle interior picture includes a first person and a second person, and records facial information and body information of the first person and the second person; A face coordinate acquisition module, used for acquiring the face coordinates of the first person and the face coordinates of the second person through a face detector; The character relationship recognition module is used to use the character relationship recognition model to process the target in-vehicle picture, the facial coordinates of the first character and the facial coordinates of the second character to obtain the character relationship recognition result between the first character and the second character, specifically including: assuming that the pixel size of the target in-vehicle picture is M*N; using the feature extraction network to extract features of the target in-vehicle picture to obtain a first feature map with a pixel size of M1*N1; converting the facial coordinates of the first character and the facial coordinates of the second character into the first feature map in proportion, and extracting two second feature maps with area sizes of Wd*Hd and Wp*Hp from them; using the feature extraction network to extract features of the two second feature maps respectively to obtain two third feature maps; using linear interpolation to convert the two third feature maps The feature maps are adjusted to the same pixel size, and a fully connected network is used to extract feature vectors of dimension V from the two third feature maps after the pixel size is adjusted; matrix calculation is performed on the two extracted feature vectors to obtain a weight matrix of dimension V*V for characterizing whether the facial emotions between the first person and the second person are similar, and the weight matrix is ​​interpolated to obtain a weight feature map with a pixel size of M1*N1; the weight feature map is added to the first feature map to obtain a new first feature map with a pixel size of M1*N1; the feature extraction network is used to extract features from the new first feature map, and the sigmoid function is used to classify the features extracted from the new first feature map to obtain a character relationship recognition result between the first person and the second person.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images