Personnel detection method and device, electronic equipment, storage medium and vehicle

By combining the first detection model and the second detection model, human body data and face data are identified and matched with key points of the human body, the accuracy of personnel detection in complex postures is solved, the error rate is reduced, and the detection accuracy is improved.

CN119027972APending Publication Date: 2024-11-26BEIJING CO WHEELS TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310589986.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-24
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The prior art is difficult to accurately identify people in complex human postures, resulting in a high rate of personnel detection errors.

Method used

Using a combination of two detection models, the first detection model is used to detect human body data, head data and face data, the second detection model is used to detect key points of the human body, and records the position of the human body through matching, which is suitable for various human bodies. posture.

Benefits of technology

Reduces false detection, improves the accuracy of personnel detection, and can identify personnel positions in complex postures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027972A_ABST
    Figure CN119027972A_ABST
Patent Text Reader

Abstract

The invention relates to a personnel detection method and device, electronic equipment, a storage medium and a vehicle, and particularly relates to the technical field of vehicle-mounted electronics. The method comprises the steps that a to-be-detected image is acquired, the to-be-detected image is input into a first detection model and a second detection model, the first detection model is used for detecting at least one of human body data, human head data and human face data in the image, and the second detection model is used for detecting human body key points in the image; obtaining at least one of human body data, head data and face data output by the first detection model and human body key points output by the second detection model; and under the condition that the human body key points are matched with at least one of the human body data, the human head data and the human face data, recording personnel positions corresponding to the human body key points. The embodiment of the invention is used for solving the problem of high personnel detection error rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle-mounted electronic technology, and in particular to a personnel detection method, device, electronic equipment, storage medium and vehicle. Background Art

[0002] Knowing the occupant status of each seat in the vehicle cabin is a prerequisite for the realization of many intelligent functions. At present, the human body data or head data of the driver and passenger is mainly identified by the model, and the position of the driver and passenger is determined according to the position of the human body data or head data. However, in actual application scenarios, the postures of the driver and passenger are more complicated. In addition to the correct sitting posture, there are various other possible postures. For example, the passenger lies on the seat. The existing model is difficult to accurately identify complex postures through single human body data or head data, resulting in a high error rate in personnel detection. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a personnel detection method, device, electronic device, storage medium and vehicle, which can accurately identify complex postures and reduce the error rate of personnel detection.

[0004] In order to achieve the above objectives, the technical solutions provided by the embodiments of the present disclosure are as follows:

[0005] In a first aspect, the present disclosure provides a method for detecting a person, comprising:

[0006] Acquire an image to be detected, and input the image to be detected into a first detection model and a second detection model, wherein the first detection model is used to detect at least one of human body data, head data and face data in the image, and the second detection model is used to detect key points of the human body in the image;

[0007] Acquire at least one of the human body data, human head data and human face data output by the first detection model, and the human body key points output by the second detection model;

[0008] When the key point of the human body matches at least one of the human body data, the head data and the face data, the position of the person corresponding to the key point of the human body is recorded.

[0009] As an optional implementation of the embodiment of the present disclosure, obtaining an image to be detected and inputting the image to be detected into a first detection model and a second detection model includes:

[0010] Acquire an image to be detected, and input the image to be detected into a first detection model;

[0011] According to the human body data output by the first detection model, cropping the image to be detected to obtain a human body region image;

[0012] The human body region image is input into the second detection model.

[0013] As an optional implementation of the embodiment of the present disclosure, when a key point of a human body matches at least one of the human body data, the head data, and the face data, recording the position of a person corresponding to the key point of the human body includes:

[0014] Determine whether the confidence of the head data and / or the face data is greater than or equal to a preset threshold, the confidence indicating the probability that the image to be detected contains the head data and / or the face data;

[0015] In the case where the human body data matches the head data and / or the face data, and in the case where the confidence of the head data and / or the face data is less than a preset threshold, determine the target external bounding box corresponding to the human body key points, as well as the head external bounding box corresponding to the head data, and / or the face external bounding box corresponding to the face data; wherein the target external bounding box is a first external bounding box corresponding to the head region key points among the human body key points, and / or a second external bounding box corresponding to the face region key points among the human body key points;

[0016] Performing intersection-over-union matching on the target bounding box, the head bounding box and / or the face bounding box;

[0017] When the intersection-union matching is successful, the position of the person corresponding to the key points of the human body is recorded.

[0018] As an optional implementation of the embodiment of the present disclosure, when the human body data matches the head data and / or the face data, after determining whether the confidence of the head data and / or the face data is greater than or equal to a preset threshold, the method further includes:

[0019] When the confidence level of the head data and / or the face data is greater than or equal to a preset threshold, determining a key point of a hip region among key points of the human body;

[0020] Record the position of the person corresponding to the key points in the hip area.

[0021] As an optional implementation of the embodiment of the present disclosure, when the key point of the human body matches at least one of the human body data, the head data and the face data, after recording the position of the person corresponding to the key point of the human body, the method further includes:

[0022] Obtaining regional location data of the vehicle cabin environment;

[0023] By comparing the regional position data, the seat area corresponding to the personnel position is determined to obtain the personnel occupancy result.

[0024] As an optional implementation of the embodiment of the present disclosure, obtaining an image to be detected and inputting the image to be detected into a first detection model and a second detection model includes:

[0025] After acquiring the image to be detected, the image to be detected is preprocessed, and the preprocessing includes image cropping and image enhancement;

[0026] The preprocessed image to be detected is input into the first detection model and the second detection model.

[0027] In a second aspect, the present disclosure provides a personnel detection device, comprising:

[0028] An acquisition module, used for acquiring an image to be detected, and inputting the image to be detected into a first detection model and a second detection model, wherein the first detection model is used for detecting at least one of human body data, head data and face data in the image, and the second detection model is used for detecting key points of the human body in the image;

[0029] A model detection module, used to obtain at least one of the human body data, head data and face data output by the first detection model, and the human body key points output by the second detection model;

[0030] The recording module is used to record the position of the person corresponding to the key point of the human body when the key point of the human body matches at least one of the human body data, the head data and the face data.

[0031] As an optional implementation of the embodiment of the present disclosure, the acquisition module is specifically used to: acquire the image to be detected and input the image to be detected into the first detection model; crop the image to be detected to obtain a human body area image according to the human body data output by the first detection model; and input the human body area image into the second detection model.

[0032] As an optional implementation of the embodiment of the present disclosure, the recording module is specifically used to determine whether the confidence of the head data and / or the face data is greater than or equal to a preset threshold when the human body data matches the head data and / or the face data, and the confidence indicates the probability that the image to be detected contains the head data and / or the face data;

[0033] In the case where the human body data matches the head data and / or the face data, and in the case where the confidence of the head data and / or the face data is less than a preset threshold, determine the target external bounding box corresponding to the human body key points, as well as the head external bounding box corresponding to the head data, and / or the face external bounding box corresponding to the face data; wherein the target external bounding box is a first external bounding box corresponding to the head region key points among the human body key points, and / or a second external bounding box corresponding to the face region key points among the human body key points;

[0034] Performing intersection-over-union matching on the target bounding box, the head bounding box and / or the face bounding box;

[0035] When the intersection-union matching is successful, the position of the person corresponding to the key points of the human body is recorded.

[0036] As an optional implementation of the embodiment of the present disclosure, the recording module, when the human body data matches the head data and / or the face data, after determining whether the confidence of the head data and / or the face data is greater than or equal to a preset threshold, is further used to: determine the hip area key points among the human body key points when the confidence of the head data and / or the face data is greater than or equal to the preset threshold; and record the personnel position corresponding to the hip area key points.

[0037] As an optional implementation of the embodiment of the present disclosure, the recording module is also used to: obtain regional position data of the vehicle cabin environment; and determine the seat area corresponding to the personnel position by comparing the regional position data to obtain the personnel occupancy result.

[0038] As an optional implementation of the embodiment of the present disclosure, the acquisition module is specifically used to: after acquiring the image to be detected, preprocess the image to be detected, and the preprocessing includes image cropping and image enhancement;

[0039] The preprocessed image to be detected is input into the first detection model and the second detection model.

[0040] In a third aspect, the present disclosure provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the personnel detection method as described in the first aspect or any optional embodiment thereof is implemented.

[0041] In a fourth aspect, the present disclosure provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the personnel detection method as described in the first aspect or any optional embodiment thereof is implemented.

[0042] In a fifth aspect, the present disclosure provides a vehicle, comprising: a personnel detection device as described in the second aspect or any optional embodiment, or an electronic device as described in the third aspect.

[0043] In a sixth aspect, a computer program product is provided, comprising: when the computer program product is run on a computer, the computer is enabled to implement the personnel detection method as described in the first aspect or any optional embodiment thereof.

[0044] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0045] The personnel detection method provided by the embodiment of the present disclosure first obtains an image to be detected, and inputs the image to be detected into a first detection model and a second detection model, wherein the first detection model is used to detect at least one of the human body data, head data and face data in the image, and the second detection data is used to detect the human body key points in the image, and then obtains at least one of the human body data, head data and face data output by the first detection model, and obtains the human body key points output by the second detection model; when the human body key points match at least one of the human body data, head data and face data, it indicates that the person is detected, and then the person position corresponding to the human body key points is recorded, so that the output of the first detection model is matched with the output of the second detection model, and the human body key points are compared with at least one of the human body data, head data and face data. This method is applicable to various human postures, can reduce false detections, and improves the accuracy of personnel detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 A flowchart of a method for detecting a person provided by an embodiment of the present disclosure;

[0049] Figure 2 A schematic diagram of the training process of the first detection model provided in an embodiment of the present disclosure;

[0050] Figure 3 A schematic diagram of the training process of the second detection model provided in the embodiment of the present disclosure;

[0051] Figure 4 A schematic diagram of the structure of a person detection device is provided for an embodiment of the present disclosure;

[0052] Figure 5 A schematic structural diagram of an electronic device is provided in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION

[0053] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0055] At present, the human body data or head data of the driver and passenger are mainly identified by the model, and the position of the driver and passenger is determined based on the position of the human body data or head data. However, in actual application scenarios, the postures of the driver and passenger are relatively complex. In addition to the correct sitting posture, there are various other possible postures. For example, the passenger lies on the seat, which causes the position of the human body data or head data detected by the model to be inconsistent with the actual position of the passenger. It can be seen that the existing model is difficult to accurately identify complex postures through single human body data or head data, resulting in a high error rate in personnel detection.

[0056] In order to solve the above problems, the embodiments of the present disclosure provide a personnel detection method, device, electronic device, storage medium and vehicle, wherein the method first obtains an image to be detected, and inputs the image to be detected into a first detection model and a second detection model, wherein the first detection model is used to detect at least one of the human body data, head data and face data in the image, and the second detection model is used to detect the human body key points in the image, and obtain at least one of the human body data, head data and face data output by the first detection model, and the human body key points output by the second detection model; when the human body key points match at least one of the human body data, head data and face data, it indicates that a person is detected, and then the person position corresponding to the human body key points is recorded, so that the output of the first detection model is matched with the output of the second detection model, and the human body key points are compared with at least one of the human body data, head data and face data. It is applicable to various human postures, can reduce false detection, and improve the accuracy of personnel detection.

[0057] The personnel detection method provided in the embodiments of the present disclosure can be implemented by a personnel detection device or an electronic device, and the device may include: a mobile phone (such as a folding screen mobile phone, a smart phone, etc.), a tablet computer, a laptop computer, a vehicle-mounted terminal, etc. The present disclosure does not impose any restrictions on the specific type of electronic device. Among them, the present disclosure does not limit the type of operating system of the electronic device. For example, Android system, Linux system, Windows system or iOS system, etc.

[0058] like Figure 1 As shown, Figure 1 A flow chart of a method for detecting a person provided in an embodiment of the present disclosure is provided. The method comprises the following steps S101 to S103:

[0059] S101, obtaining an image to be detected, and inputting the image to be detected into a first detection model and a second detection model.

[0060] The image to be detected is an image collected by a camera configured in the vehicle cockpit environment.

[0061] In some embodiments, after acquiring the image to be detected, the image to be detected is preprocessed, including but not limited to image cropping and image enhancement. The present disclosure does not limit the preprocessing. Further, the preprocessed image to be detected is input into the first detection model and the second detection model.

[0062] The first detection model is used to detect at least one of human body data, head data and face data in the image. Optionally, the output of the first detection model includes human body data and head data, or human body data and face data, or human body data, head data and face data; the second detection model is used to detect key points of the human body in the image. Both the first detection model and the second detection model are pre-trained convergent detection models.

[0063] The following will introduce the training process of the first detection model and the second detection model respectively:

[0064] (1) Training process of the first detection model

[0065] like Figure 2 As shown, Figure 2 The training process diagram of the first detection model provided in the embodiment of the present disclosure includes the following steps S201 to S204:

[0066] S201: Obtain a first training data set.

[0067] Among them, the first training data set includes: multiple frames of first-person feature images, and a first annotated image corresponding to each frame of the first-person feature image, the first annotated image is annotated with at least one of human body data, head data and facial data based on the first-person feature image, and optionally, human body data and head data, or human body data and facial data, or an image of human body data, head data and facial data are annotated on the first-person feature image.

[0068] In some embodiments, human body data includes: a human body bounding box annotated on the basis of the image to be detected, and coordinate information of the human body bounding box. Human head data includes: a human head bounding box annotated on the basis of the image to be detected, and coordinate information of the human head bounding box. Human face data includes: a human face bounding box annotated on the basis of the image to be detected, and coordinate information of the human face bounding box.

[0069] In some embodiments, in order to improve the training accuracy, the multi-frame first-person feature images provided by the present disclosure include partially coded faces. The present disclosure provides an implementation method, after obtaining the multi-frame first-person feature images, a random algorithm is used to randomly extract partial person feature images from the multi-frame first-person feature images, and these partial person feature images are coded to obtain partially coded faces. The first-person feature images other than the partially coded faces in the multi-frame first-person feature images can be the original first-person feature images.

[0070] Optionally, a face coding image refers to an image in which the grayscale value of all pixels within the face external frame is 0. After obtaining multiple frames of first-person feature images, for any first-person feature image in the multiple frames, the face is identified as a region of interest. After the region of interest is identified, the region of interest is marked in the first-person feature image to obtain a face external frame. At this time, the first-person feature image is divided into two parts: the image within the face external frame and the image outside the face external frame. Then, the grayscale value of the image within the face external frame is adjusted to 0, so that the grayscale values ​​of all pixels within the face external frame in the first-person feature image are 0, thereby achieving face coding.

[0071] Optionally, after randomly extracting a partial human feature image, extract facial image features of the partial human feature image, including but not limited to: facial area features, eye area features, and lip area features. The disclosed embodiment may encode one or more of the above facial image features. Exemplarily, the disclosed embodiment encodes the extracted eye area features to obtain a partially encoded face image.

[0072] Optionally, after randomly extracting some human feature images, when coding the partial human feature images, the face area of ​​the partial human feature images can be covered by a pre-set mask image to obtain a partially coded face image, thereby enhancing the diversity of sample data and improving the robustness of model training. The parameters such as the size and grayscale value of the mask image are not specifically limited in this disclosure.

[0073] The above optional implementation modes may be combined arbitrarily. For example, after randomly extracting a partial human feature image, the eye area features of the partial human feature image are extracted, and the eye area features in the partial human feature image are covered by a mask image corresponding to a pre-set eye area to obtain a partial face coded image.

[0074] The above embodiment codes part of the first-person feature images included in the acquired multiple frames of first-person feature images, thereby laying a foundation for subsequent training and updating of the model using online data, thereby improving the robustness and accuracy of the model training.

[0075] In some embodiments, the multiple frames of first-person feature images provided by the present disclosure include partially coded images of human heads. The embodiments of the present disclosure provide an implementation method. After capturing multiple frames of first-person feature images, a random algorithm is used to randomly extract partial human feature images from the multiple frames of first-person feature images, and these partial human feature images are coded to obtain partially coded images of human heads. The other first-person feature images in the multiple frames of first-person feature images except the partially coded images of human heads may be the original first-person feature images and / or partially coded images of human faces.

[0076] Optionally, the head coding image refers to an image in which the grayscale values ​​of all pixels inside the head external frame are 0. After obtaining multiple frames of first-person feature images, for any first-person feature image in the multiple frames, the head is identified as the region of interest. After the region of interest is identified, the region of interest is marked in the first-person feature image to obtain the head external frame. At this time, the first-person feature image is divided into two parts: the image inside the head external frame and the image outside the head external frame. Then, the grayscale value of the image inside the head external frame is adjusted to 0, so that the grayscale values ​​of all pixels inside the head external frame in the first-person feature image are 0, thereby achieving head coding.

[0077] Optionally, after randomly extracting a partial human feature image, extract the head image features of the partial human feature image, including but not limited to: head region features, ear region features, and facial region features. The disclosed embodiment may code one or more of the above head image features. Exemplarily, the disclosed embodiment codes the extracted head region features to obtain a partially coded head image.

[0078] Optionally, after randomly extracting some of the human feature images, when coding the partial human feature images, the head area of ​​the partial human feature images can be covered by a pre-set mask image to obtain a partial head coded image. The parameters such as the size and grayscale value of the mask image are not specifically limited in the present disclosure. It is understandable that the mask image can partially cover the head area or completely cover the head area.

[0079] The above optional implementation modes may be combined arbitrarily. For example, after randomly extracting a partial person feature image, the ear area features of the partial person feature image are extracted, and the ear area features in the partial person feature image are covered by a mask image corresponding to a preset ear area to obtain a partial person head coded image.

[0080] In some embodiments, the multi-frame first-person feature images provided by the present disclosure include human body coding images. In the images captured in the actual driving environment, there are cases where the cabin interior obscures the occupants, such as the seat headrest obscures the rear passengers. Human body coding is adapted to such actual obstruction scenarios, and the trained model has strong applicability and high accuracy.

[0081] The disclosed embodiment provides an implementation method, after capturing multiple frames of first person feature images, a random algorithm is used to randomly extract partial person feature images from the multiple frames of first person feature images, and these partial person feature images are coded to obtain partial human body coded images. The other first person feature images in the multiple frames of first person feature images except the partial human body coded images may be at least one of: the original first person feature image, a partial human face coded image, and a partial human head coded image.

[0082] Optionally, a human body coding image refers to an image in which the grayscale values ​​of all pixels inside the human body external frame are 0. After obtaining multiple frames of first-person feature images, for any first-person feature image in the multiple frames of first-person feature images, the human body is identified as a region of interest. After the region of interest is identified, the region of interest is marked in the first-person feature image to obtain a human body external frame. At this time, the first-person feature image is divided into two parts: an image inside the human body external frame and an image outside the human body external frame. Then, the grayscale value of the image inside the human body external frame is adjusted to 0, so that the grayscale values ​​of all pixels inside the human body external frame in the first-person feature image are 0, thereby achieving human body coding.

[0083] Optionally, after randomly extracting a partial human feature image, human image features of the partial human feature image are extracted, including but not limited to: human body region features, fingerprint region features, and palm print region features. The disclosed embodiment may code one or more of the above human image features. Exemplarily, the disclosed embodiment codes the extracted palm print region features to obtain a partial human body coded image.

[0084] Optionally, after randomly extracting a partial human feature image, when coding the partial human feature image, the human body region of the partial human feature image can be covered by a pre-set mask image to obtain a partial human body coded image. The parameters such as the size and grayscale value of the mask image are not specifically limited in the present disclosure. It is understood that the mask image can partially cover the human body region or completely cover the human body region.

[0085] The above optional implementation modes may be combined arbitrarily. For example, after randomly extracting a partial human feature image, the palm print area features of the partial human feature image are extracted, and the palm print area features in the partial human feature image are covered by a mask image corresponding to a preset palm print area to obtain a partial human body coded image.

[0086] In some embodiments, the multiple frames of first person feature images provided by the present disclosure include at least one of the above-mentioned coded images, that is, the multiple frames of first person feature images include: at least one of: a face coded image, a head coded image, and a body coded image. Accordingly, the first training data set includes annotated images corresponding to each coded image.

[0087] The above embodiment desensitizes the biometric information based on the first training data set collected manually by coding at least one of the face, head and body, so as to be suitable for desensitizing the online data when the online data is subsequently used for training and updating the model.

[0088] S202: Based on the first training data set, train the initial detection model to obtain a pending detection model.

[0089] The initial detection model is based on at least one neural network of a backbone network, a neck network, and a head network. The pending detection model is used to detect one or more of human body data, head data, and face data in an image.

[0090] During the training process, multiple frames of first-person feature images in the first training data set are input into the initial detection model, and the initial detection model outputs a predicted annotated image. The predicted annotated image is compared with the first annotated image in the first training data set, and the loss function value is calculated. When the loss function value is greater than the preset loss value, the model parameters of the initial detection model are returned to adjust until the loss function value between the predicted annotated image output by the model and the first annotated image is less than or equal to the preset loss function value. The model converges, and a converged pending detection model is obtained.

[0091] S203: Acquire multiple frames of second person feature images collected in the vehicle cabin environment from online data.

[0092] Among them, the online data is the image data captured after the user's permission, including several second-person feature images collected in the vehicle cabin environment in the actual scene.

[0093] In some embodiments, the plurality of frames of second-person feature images include: all second-person feature images of a target scene, and at least one frame of second-person feature images of other scenes. The target scene includes: a scene of a special group of people, and / or a dangerous scene. The target scene and other scenes can be distinguished by manual annotation. Exemplarily, the target scene can be a scene of passengers moving freely in the aisle of the cabin, or a scene of an infant leaving a child safety seat.

[0094] The disclosed embodiment provides an implementation method for obtaining multiple frames of second person feature images collected in a vehicle cockpit environment from online data, including: obtaining several frames of second person feature images collected in a vehicle cockpit environment from online data, and then determining all second person feature images of a target scene and all second person feature images of other scenes from these several frames of second person feature images, and then randomly sampling from all second person feature images of other scenes to obtain at least one frame of second person feature images of other scenes. Thus, at least one frame of second person feature images of other scenes is used as a negative sample for model training, and then the human feature detection model is trained subsequently to accurately distinguish the target scene from other scenes.

[0095] In some embodiments, after obtaining multiple frames of second-person feature image packages collected in a vehicle cockpit environment from online data, data cleaning is first performed on the multiple frames of second-person feature image packages. Optionally, feature extraction is performed on the multiple frames of second-person feature image packages containing noise based on a neural network model pre-trained on an image dataset (ImageNet). Unsupervised clustering algorithms such as density-based spatial clustering of applications with noise (DBSCAN) are used to cluster the second-person feature image packages based on the extracted features to obtain multiple clusters, determine the cluster containing the largest number of image data, and use the image data in the cluster as clean data after cleaning. Further, data annotation is performed on the clean data, that is, the second-person feature image package after data cleaning.

[0096] In some embodiments, data annotation is performed on multiple frames of second-person feature images, including annotating human body data and head data, or human body data and facial data, or human body data, head data and facial data, as well as annotating target scenes or other scenes. For example, for any second-person feature image in multiple frames of second-person feature images, the face is identified as a region of interest, and after the region of interest is identified, the region of interest is annotated in the second-person feature image to obtain a face bounding box, and the coordinate information of the face bounding box is determined. The annotation of head data and human body data is similar, and the present disclosure will not elaborate on this. For any second-person feature image in multiple frames of second-person feature images, if it is determined that it corresponds to a target scene, it is labeled with the target scene; if it is determined that it corresponds to other scenes, it is labeled with other scenes.

[0097] Furthermore, the annotated multiple frames of the second person feature images are coded to protect user privacy. The coding process includes coding the face, coding the head, and coding the body. Accordingly, the multiple frames of the second person feature images can be at least one of the second face coded image, the second head coded image, and the second body coded image. The following will introduce these three coding methods respectively:

[0098] (1) Face coding

[0099] The multiple frames of the second person feature images include a second person face coding image.

[0100] Optionally, after the second person feature image is face-labeled, the second person feature image is divided into two parts: an image within the face circumference frame and an image outside the face circumference frame. Then, the grayscale value of the image within the face circumference frame is adjusted to 0, so that the grayscale values ​​of all pixels within the face circumference frame in the second person feature image are 0, thereby achieving face coding.

[0101] Optionally, for any second person feature image in the multiple frames of second person feature images, extract facial image features of the second person feature image, including but not limited to: facial area features, eye area features, and lip area features. The disclosed embodiment may encode one or more of the above facial image features. Exemplarily, the disclosed embodiment encodes the extracted eye area features to obtain a second face encoded image.

[0102] Optionally, for any second person feature image in the multiple frames of second person feature images, the face area of ​​the second person feature image can be masked by a pre-set mask image to obtain a second face coded image, thereby enhancing the diversity of sample data and improving the robustness of model training. The parameters such as the size and grayscale value of the mask image are not specifically limited in this disclosure.

[0103] The above optional implementation modes may be combined arbitrarily. For example, for any second person feature image in multiple frames of second person feature images, the eye area features of the second person feature image are extracted, and the eye area features in the second person feature image are covered by a mask image corresponding to a preset eye area to obtain a second face coded image.

[0104] The above embodiment avoids leaking the user's facial privacy information during model training by coding the face of the acquired multiple frames of second person feature images, so that the effect of desensitization detection can be achieved when the model is subsequently applied for portrait detection.

[0105] (2) Head coding

[0106] The multiple frames of the second person feature images include: a second head coded image, and in addition, may also include a second face coded image.

[0107] Optionally, after the second person feature image is labeled with a head, the second person feature image is divided into two parts: an image within the head external frame and an image outside the head external frame. Then, the grayscale value of the image within the head external frame is adjusted to 0, so that the grayscale values ​​of all pixels within the head external frame in the second person feature image are 0, thereby achieving head coding.

[0108] Optionally, for any second person feature image in the multiple frames of second person feature images, extract head image features of the second person feature image, including but not limited to: head region features, ear region features, and facial region features. The disclosed embodiment may code one or more of the above head image features. Exemplarily, the disclosed embodiment codes the extracted head region features to obtain a second head coded image.

[0109] Optionally, for any second person feature image in the multiple frames of second person feature images, the head area of ​​the second person feature image can be covered by a pre-set mask image to obtain a second head coded image. The parameters such as the size and grayscale value of the mask image are not specifically limited in the present disclosure. It is understandable that the mask image can partially cover the head area or completely cover the head area.

[0110] The above optional implementation modes may be combined arbitrarily. For example, for any second person feature image in multiple frames of second person feature images, the ear area features of the second person feature image are extracted, and the ear area features in the second person feature image are covered by a mask image corresponding to a preset ear area to obtain a second head coded image.

[0111] (3) Human body coding

[0112] The multiple frames of second person feature images include a second human body coded image, and in addition may also include a second face coded image and / or a second head coded image.

[0113] Optionally, after the second person feature image is annotated with human body, the second person feature image is divided into two parts: an image within the human body external frame and an image outside the human body external frame. Then, the grayscale value of the image within the human body external frame is adjusted to 0, so that the grayscale values ​​of all pixels within the human body external frame in the second person feature image are 0, thereby realizing human body coding.

[0114] Optionally, after randomly extracting the second person feature image, extract the human image features of the second person feature image, including but not limited to: human body region features, fingerprint region features, and palm print region features. The disclosed embodiment may code one or more of the above human image features. Exemplarily, the disclosed embodiment codes the extracted palm print region features to obtain the second human coded image.

[0115] Optionally, after randomly extracting the second person feature image, when coding the second person feature image, the human body region of the second person feature image may be covered by a pre-set mask image to obtain the second human body coded image. The parameters such as the size and grayscale value of the mask image are not specifically limited in the present disclosure. It is understood that the mask image may partially cover the human body region or completely cover the human body region.

[0116] The above optional implementation modes may be combined arbitrarily. For example, after randomly extracting the second person feature image, the palm print area features of the second person feature image are extracted, and the palm print area features in the second person feature image are covered by a mask image corresponding to a pre-set palm print area to obtain a second human body coded image.

[0117] Furthermore, multiple frames of second person feature images after coding and second annotated images corresponding to each frame of the second person feature image are used as a second training data set, wherein the second annotated image is an image in which human body data and head data, or human body data and facial data, or human body data, head data and facial data are annotated on the basis of the second person feature image, and one or more of the specific human body features, head features and facial features in the second annotated image have been coded and are desensitized.

[0118] S204: Based on the second training data set, update the model parameters of the to-be-determined detection model to obtain a first detection model.

[0119] The second training data set includes: multiple frames of second person feature images, and second annotated images corresponding to each frame of the second person feature image.

[0120] Among them, the second training data set includes multiple frames of second person feature images, and the second annotated image corresponding to each frame of the second person feature image. Each frame of the second person feature image is a person feature image after coding, and can be at least one of a second face coding image, a second head coding image, and a second body coding image. Optionally, each frame of the second person feature image is a second face coding image; the pending detection model is obtained by training based on a portion of the face coding image. Similarly, each frame of the second person feature image is a second head coding image or a second body coding image, and the corresponding pending detection model is obtained by training based on a portion of the head coding image or a portion of the body coding image.

[0121] In some embodiments, based on the second training data set, the pending detection model is updated to obtain the first detection model, including using multiple frames of second person feature images in the second training data set as inputs to the pending detection model to obtain a predicted annotated image output by the pending detection model, comparing the predicted annotated image output by the pending detection model with the second annotated image, calculating the loss function value between the predicted annotated image and the second annotated image according to a loss function, and when the loss function value is greater than a preset loss value, returning to adjust the model parameters of the pending detection model until the loss function value between the predicted annotated image output by the pending detection model and the second annotated image is less than or equal to the preset loss value, and determining that the pending detection model converges, thereby achieving an update of the pending detection model to obtain a converged first detection model. The first detection model can perform portrait detection on coded images.

[0122] (2) Training process of the second detection model

[0123] like Figure 3 As shown, Figure 3 The training process diagram of the second detection model provided in the embodiment of the present disclosure includes the following steps S301 to S304:

[0124] S301: Obtain a third training data set.

[0125] The third training data set includes: multiple frames of first person feature images, and third annotated images corresponding to each frame of third person feature images, and the third annotated images are images with key points of the human body annotated on the basis of the third person feature images. Key points of the human body are points corresponding to key parts of the human body, such as nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle and other key parts. Annotating key points of the human body includes annotating the types and coordinates of the key points of the human body.

[0126] It should be noted that the data desensitization process for the third training data set is the same or similar to the data desensitization process for the first training data set (including but not limited to face coding), and the present disclosure will not elaborate on it here. It is understandable that in some embodiments, the third training data set includes the original first person feature image, the partially coded face image, and the third annotated image corresponding to the first person feature image and the third annotated image corresponding to the partially coded face image, and the third annotated image corresponding to the partially coded face image is an image with key points of the human body annotated on the basis of the partially coded face image.

[0127] S302: Based on the third training data set, train the pre-trained detection model to obtain an intermediate detection model.

[0128] Among them, the pre-trained detection model is based on a human key point detection network and is used to detect human key points in images.

[0129] During training, multiple frames of third-person feature images in the third training data set are input into the pre-trained detection model, and the pre-trained detection model outputs an inferred annotated image. The inferred annotated image is compared with the third annotated image in the third training data set, and the loss function value is calculated. When the loss function value is greater than the preset loss value, the model parameters of the pre-trained detection model are returned to adjust until the loss function value between the inferred annotated image output by the model and the third annotated image is less than or equal to the preset loss value. It can be determined that the training of the pre-trained detection model has converged, and a converged intermediate detection model is obtained.

[0130] S303 , acquiring multiple frames of second person feature images collected in the vehicle cockpit environment from online data.

[0131] The implementation of step S303 is the same as or similar to the implementation of step S203 described above, and is not described in detail herein.

[0132] S304: Based on the fourth training data set, update the intermediate detection model to obtain a second detection model.

[0133] The fourth training data set includes: multiple frames of second person feature images, and a fourth annotated image corresponding to each frame of the second person feature image, where the fourth annotated image is an image with key points of the human body annotated on the basis of the second person feature image.

[0134] In some embodiments, in the process of updating the intermediate detection model to obtain the second detection model based on the fourth training data set, multiple frames of second person feature images in the fourth training data set are input into the intermediate detection model to obtain a labeled image output by the intermediate detection model, and the labeled image is compared with the fourth labeled image to calculate the loss function value between the two, and it is determined whether the loss function value is less than or equal to the preset loss value. If so, it is determined that a converged intermediate detection model is obtained, and the converged intermediate detection model is used as the second detection model, which can detect key points of the human body for the coded image.

[0135] It should be noted that, during the training process of the first detection model and the second detection model, the same multiple frames of first person feature images can be obtained, but the difference is the annotation method. For the first detection model, human body data and head data, or human body data and face data, or human body data, head data and face data are annotated on each frame of the first person feature image to obtain a first annotated image; while for the second detection model, human body key points are annotated on each frame of the first person feature image to obtain a third annotated image.

[0136] In some embodiments, after the first detection model and the second detection model are trained to converge, the present disclosure inputs the image to be detected into the first detection model and the second detection model to obtain the detection results output by the first detection model and the second detection model.

[0137] In some embodiments, the image to be detected is first input into the first detection model, and the first detection model outputs human body data and head data, or human body data and face data, or human body data, head data and face data; then the image to be detected is cropped according to the human body data to obtain a human body region image, specifically, the image to be detected is cropped according to the human body external bounding box corresponding to the human body data, and the image within the human body external bounding box is retained as the human body region image. Furthermore, the human body region image is input into the second detection model to obtain the human body key points output by the second detection model, thereby deleting image information in the image to be detected that is not related to the human body key points, so that the detection result of the second detection model is more accurate.

[0138] S102. Obtain at least one of the human body data, head data and face data output by the first detection model, and the human body key points output by the second detection model.

[0139] In some embodiments, the first detection model outputs: human body data and human head data, or human body data and human face data, or human body data, human head data and human face data.

[0140] S103. When the key point of the human body matches at least one of the human body data, the head data and the face data, the position of the person corresponding to the key point of the human body is recorded.

[0141] In some embodiments, when judging whether the key points of the human body match at least one of the human body data, head data and face data, the following three situations can be considered: ① whether the human body data matches the head data; ② whether the human body data matches the face data; ③ whether the human body data matches the head data and the face data. It should be emphasized that the key points of the human body are specifically used in the different matching results of the above three situations.

[0142] The following will introduce these three situations:

[0143] ①Whether the human body data matches the head data

[0144] In some embodiments, after obtaining the human body data and human head data output by the first detection model, the human body data and the human head data are matched using the Hungarian algorithm. Specifically, the human body external frame and the head external frame are matched using the Hungarian algorithm to correspond the human body data and the human head data.

[0145] If the human body data and the head data match successfully, the confidence of the head data is further determined. The confidence of the head data is used to indicate the probability that the image to be detected contains the head data, which is calculated and output by the first detection model. Then, it is determined whether the confidence of the head data is greater than or equal to a preset threshold value, which is a preset minimum probability for determining that the image contains the head data.

[0146] If the confidence of the head data is greater than or equal to the preset threshold, it means that the image to be detected contains the head data. It can be understood that there is a valid human body corresponding to the head data and the matching human body data, and the person position corresponding to the human body key point is recorded.

[0147] If the confidence of the head data is less than the preset threshold, it is preliminarily determined that the image to be detected does not contain the head data, then the first external bounding box corresponding to the head area key point and the head external bounding box corresponding to the head data are determined according to the head area key point in the human key point output by the second detection model, and then the head external bounding box is matched with the first external bounding box by intersection and union ratio. If the intersection and union ratio match between the head external bounding box and the first external bounding box is successful, it means that after the secondary processing, it is determined that the image to be detected contains the head data, which means that there is a valid human body corresponding to the head data and the human body data matched therewith; further, the position of the person corresponding to the human body key point is recorded.

[0148] If the human body data and the human head data fail to match, it is determined that the detected human body data is a misjudgment and the human body data is deleted.

[0149] In other embodiments, the human body data, human head data and human body key points are matched. Specifically, the human body external frame is matched with the minimum external frame of the human body key points, and the human head external frame is matched with the first external frame corresponding to the key points of the head area among the human body key points. If all matches are successful, it is determined that there is a valid human body corresponding to the human body data, and the person position corresponding to the human body key points is recorded, that is, the coordinates of the human body key points are used as the person position coordinates.

[0150] ②Whether the human body data matches the face data

[0151] In some embodiments, after obtaining the human body data and facial data output by the first detection model, the human body data and the facial data are matched using the Hungarian algorithm. Specifically, the human body bounding box and the facial bounding box are matched using the Hungarian algorithm to correspond the human body data and the facial data.

[0152] If the human body data and the face data match successfully, the confidence of the face data is further determined. The confidence of the face data is used to indicate the probability that the image to be detected contains face data, which is calculated and output by the first detection model. Then, it is determined whether the confidence of the face data is greater than or equal to a preset threshold, which is a preset minimum probability for determining that face data is contained.

[0153] If the confidence of the facial data is greater than or equal to the preset threshold, it means that the facial data is contained in the image to be detected. It can be understood that there is a valid human body corresponding to the facial data and the matching human body data, and the person position corresponding to the human body key points is recorded.

[0154] If the confidence of the face data is less than the preset threshold, it is preliminarily determined that the image to be detected does not contain the face data, then the second external bounding box corresponding to the face area key point and the face external bounding box corresponding to the face data are determined according to the face area key point in the human body key point output by the second detection model, and then the face external bounding box is matched with the second external bounding box by intersection and union ratio. If the intersection and union ratio of the face external bounding box and the second external bounding box is successfully matched, it means that after the secondary processing, it is determined that the image to be detected contains face data, which means that there is a valid human body corresponding to the face data and the human body data matched therewith; further, the position of the person corresponding to the human body key point is recorded.

[0155] If the body data and face data fail to match, it is determined that the detected body data is a misjudgment and the body data is deleted.

[0156] In other embodiments, the human body data, facial data and human body key points are matched. Specifically, the human body bounding box is matched with the minimum bounding box of the human body key points, and the facial bounding box is matched with the second bounding box corresponding to the key points of the facial area among the human body key points. If all matches are successful, it is determined that there is a valid human body corresponding to the human body data, and the person position corresponding to the human body key points is recorded, that is, the coordinates of the human body key points are used as the person position coordinates.

[0157] ③Whether the human body data matches the head data and face data

[0158] In some embodiments, after obtaining the human body data, head data and face data output by the first detection model, the human body data is matched with the head data and the face data through the Hungarian algorithm. Specifically, the human body external frame, the head external frame and the face external frame are matched through the Hungarian algorithm to correspond the human body data, head data and face data.

[0159] If the human body data, head data and face data match successfully, the confidence of the head data and face data is further determined, and then it is determined whether the confidence of the head data and face data is greater than or equal to a preset threshold.

[0160] If the confidence of the head data and face data is greater than or equal to the preset threshold, it means that the image to be detected contains the head data and face data. It can be understood that there is a valid human body corresponding to the head data and face data and the human body data matching them, then the person position corresponding to the human body key points is recorded.

[0161] If the confidence of the head data and face data is less than the preset threshold, it is preliminarily determined that the image to be detected does not contain the head data and face data, then the first external bounding box corresponding to the head area key point and the second external bounding box corresponding to the face area key point, as well as the head external bounding box corresponding to the head data and the face external bounding box corresponding to the face data are determined according to the head area key point and the face area key point in the human key points output by the second detection model, and then the head external bounding box is matched with the first external bounding box by intersection and union, and the face external bounding box is matched with the second external bounding box by intersection and union. When the head external bounding box successfully matches the first external bounding box by intersection and union, and the face external bounding box successfully matches the second external bounding box by intersection and union, it means that after the secondary processing, it is determined that the image to be detected contains the head data and face data, which means that there is a valid human body corresponding to the head data, face data and the human body data matched therewith; further, the position of the person corresponding to the human body key point is recorded.

[0162] If the human body data fails to match the head data and the face data, it is determined that the detected human body data is a misjudgment and the human body data is deleted.

[0163] In other embodiments, the human body data, human head data, human face data and human body key points are matched. Specifically, the human body external frame is matched with the minimum external frame of the human body key points through an intersection-and-union ratio match, the human head external frame is matched with the first external frame through an intersection-and-union ratio match, and the human head external frame is matched with the second external frame of the key points in the facial area of ​​the human body key points through an intersection-and-union ratio match. If all the matches are successful, it is determined that there is a valid human body corresponding to the human body data, and the person position corresponding to the human body key points is recorded, that is, the coordinates of the human body key points are used as the person position coordinates.

[0164] In some embodiments, in the process of recording the position of a person corresponding to the key points of the human body, the key points of the hip region among the key points of the human body are first determined, and the key points of the hip region are used to represent the position of the effective human body, thereby recording the position of the person corresponding to the key points of the hip region. By using the key points of the hips to represent the position of the person, the interference of complex sitting postures on the detection of the person is eliminated, so that not only the position of the person sitting in a correct posture can be recorded, but also the position of other people with complex sitting postures can be recorded, thereby avoiding misjudgment and improving the accuracy of the detection of the person.

[0165] In some embodiments, after recording the position of a person corresponding to key points of the human body, the present disclosure can also obtain regional position data of the vehicle cabin environment, which is used to indicate the seat area; the seat area where the person is located is determined by comparing the regional position data to obtain a person occupancy result, which can indicate whether there are people and how many people are sitting in each seat in the seat area in the vehicle cabin environment.

[0166] In summary, the embodiments of the present disclosure provide a person detection method, which first obtains an image to be detected, and inputs the image to be detected into a first detection model and a second detection model, wherein the first detection model is used to detect human body data, head data and / or face data in the image, and the second detection model is used to detect human body key points in the image, and obtains human body data, head data and / or face data output by the first detection model, and human body key points output by the second detection model; when the human body key points, human body data, head data and / or human body key points match, it indicates that a person is detected, and then the position of the person corresponding to the human body key points is recorded, so that by matching the output of the first detection model with the output of the second detection model, the human body data and head data, and / or the human body data and face data, are compared with the human body key points. This method is suitable for various human postures, can reduce false detections, and improves the accuracy of person detection.

[0167] like Figure 4 As shown, Figure 4 A schematic diagram of a human detection device is provided for an embodiment of the present disclosure, and the device includes:

[0168] An acquisition module 401 is used to acquire an image to be detected, and input the image to be detected into a first detection model and a second detection model, wherein the first detection model is used to detect at least one of human body data, head data and face data in the image, and the second detection model is used to detect key points of the human body in the image;

[0169] The model detection module 402 is used to obtain at least one of the human body data, head data and face data output by the first detection model, and the human body key points output by the second detection model;

[0170] The recording module 403 is used to record the position of the person corresponding to the key point of the human body when the key point of the human body matches at least one of the human body data, the head data and the face data.

[0171] As an optional implementation of the embodiment of the present disclosure, the acquisition module 401 is specifically used to: acquire the image to be detected and input the image to be detected into the first detection model; crop the image to be detected to obtain a human body area image according to the human body data output by the first detection model; and input the human body area image into the second detection model.

[0172] As an optional implementation of the embodiment of the present disclosure, the recording module 403 is specifically used to determine whether the confidence of the head data and / or face data is greater than or equal to a preset threshold when the human body data matches the head data and / or face data, and the confidence indicates the probability that the image to be detected contains the head data and / or face data;

[0173] When the confidence of the head data and / or the face data is less than a preset threshold, determine the target external bounding box corresponding to the human body key points, the head external bounding box corresponding to the head data, and / or the face external bounding box corresponding to the face data; wherein the target external bounding box is a first external bounding box corresponding to the head region key points among the human body key points, and / or a second external bounding box corresponding to the face region key points among the human body key points;

[0174] Performing intersection-over-union matching on the target bounding box, the head bounding box and / or the face bounding box;

[0175] When the intersection-union matching is successful, the position of the person corresponding to the key points of the human body is recorded.

[0176] As an optional implementation of the embodiment of the present disclosure, the recording module 403, after determining whether the confidence of the head data and / or the face data is greater than or equal to a preset threshold when the body data matches the head data and / or the face data, is also used to: determine the hip area key points among the body key points when the confidence of the head data and / or the face data is greater than or equal to the preset threshold; and record the personnel position corresponding to the hip area key points.

[0177] As an optional implementation of the embodiment of the present disclosure, the recording module 403 is also used to: obtain regional position data of the vehicle cabin environment; and determine the seat area corresponding to the personnel position by comparing the regional position data to obtain the personnel occupancy result.

[0178] As an optional implementation of the embodiment of the present disclosure, the acquisition module 401 is specifically used to: after acquiring the image to be detected, preprocess the image to be detected, and the preprocessing includes image cropping and image enhancement;

[0179] The preprocessed image to be detected is input into the first detection model and the second detection model. Figure 5 As shown, Figure 5 The present disclosure provides a schematic diagram of the structure of an electronic device, which includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, and when the computer program is executed by the processor, each process of the personnel detection method in the above method embodiment is implemented. The same technical effect can be achieved, and it will not be repeated here to avoid repetition.

[0180] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the personnel detection method in the above method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0181] The computer readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0182] An embodiment of the present invention provides a computer program product, which stores a computer program. When the computer program is executed by a processor, the various processes of the personnel detection method in the above method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0183] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0184] In the present disclosure, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0185] In the present disclosure, memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0186] In the present disclosure, computer-readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0187] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0188] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting a person, characterized in that: include: Acquire an image to be detected, and input the image to be detected into a first detection model and a second detection model, wherein the first detection model is used to detect at least one of human body data, head data and face data in the image, and the second detection model is used to detect key points of the human body in the image; Acquire at least one of the human body data, human head data and human face data output by the first detection model, and the human body key points output by the second detection model; When it is determined that the human body key point matches at least one of the human body data, the head data and the face data, the position of the person corresponding to the human body key point is recorded.

2. The method according to claim 1, characterized in that The step of acquiring the image to be detected and inputting the image to be detected into the first detection model and the second detection model comprises: Acquire the image to be detected, and input the image to be detected into the first detection model; According to the human body data output by the first detection model, cropping the image to be detected to obtain a human body region image; The human body region image is input into the second detection model.

3. The method according to claim 1, characterized in that When it is determined that the human body key point matches at least one of the human body data, the head data, and the face data, recording the personnel position corresponding to the human body key point includes: In the case where the human body data matches the head data and / or the face data, determining whether the confidence of the head data and / or the face data is greater than or equal to a preset threshold, the confidence indicating the probability that the image to be detected contains the head data and / or the face data; In the case where it is determined that the confidence of the head data and / or the face data is less than a preset threshold, a target external bounding box corresponding to the human body key points, a head external bounding box corresponding to the head data, and / or a face external bounding box corresponding to the face data are determined; wherein the target external bounding box is a first external bounding box corresponding to the head region key points among the human body key points, and / or a second external bounding box corresponding to the face region key points among the human body key points; Performing intersection-over-union matching on the target bounding box, the head bounding box and / or the face bounding box; When the intersection-over-union matching is successful, the position of the person corresponding to the key point of the human body is recorded.

4. The method according to claim 3, characterized in that In the case where the human body data matches the head data and / or the face data, after determining whether the confidence of the head data and / or the face data is greater than or equal to a preset threshold, the method further includes: When the confidence level of the head data and / or the face data is greater than or equal to a preset threshold, determining a key point of a hip region among the key points of the human body; The position of the person corresponding to the key points of the hip area is recorded.

5. The method according to claim 1, characterized in that In the case where it is determined that the human body key point matches at least one of the human body data, the head data and the face data, after recording the person position corresponding to the human body key point, the method further includes: Obtaining regional location data of the vehicle cabin environment; By comparing the area position data, the seat area corresponding to the personnel position is determined to obtain the personnel occupancy result.

6. The method according to claim 1, characterized in that The step of acquiring the image to be detected and inputting the image to be detected into the first detection model and the second detection model comprises: After acquiring the image to be detected, preprocessing the image to be detected, wherein the preprocessing includes image cropping and image enhancement; The preprocessed image to be detected is input into the first detection model and the second detection model.

7. A person detection device, characterized in that: include: An acquisition module, used for acquiring an image to be detected, and inputting the image to be detected into a first detection model and a second detection model, wherein the first detection model is used for detecting at least one of human body data, head data and face data in the image, and the second detection model is used for detecting key points of the human body in the image; A model detection module, used to obtain at least one of the human body data, head data and face data output by the first detection model, and the human body key points output by the second detection model; A recording module is used to record the position of the person corresponding to the key point of the human body when the key point of the human body matches at least one of the human body data, the head data and the face data.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the personnel detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the person detection method according to any one of claims 1 to 6 is implemented.

10. A vehicle, characterized in that: include: The personnel detection device as claimed in claim 7, or the electronic device as claimed in claim 8.