In-vehicle personnel attribute determination method and device, vehicle-mounted terminal and storage medium

By using human detection network model and video image timing matching and voting processing in the vehicle cockpit, the accuracy of personnel attribute detection in the vehicle is solved, the stability and accuracy of the detection results are improved, and the dependence on training data is reduced. .

CN119027918APending Publication Date: 2024-11-26BEIJING CO WHEELS TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310595578.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-24
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the complex environment of the vehicle cockpit, it is difficult for the prior art to accurately identify whether the person in the vehicle is an adult or a child, and the human detection classification network makes incorrect judgments on unusual scenarios, resulting in inaccurate attribute detection results.

Method used

By obtaining the video data of the vehicle cockpit, using the human body detection network model to detect human objects and attributes, and matching the detection results of adjacent frames according to the video image timing, voting processing is carried out to improve the accuracy of the detection results. The specific steps include matching the detection results of the human body external frame, summarizing the attribute detection results into the same human object, and matching and prediction through the Kalman filtering algorithm, and statistically determining the attribute detection results to determine the target attribute.

Benefits of technology

It significantly improves the accuracy of human attribute detection, reduces the dependence on training data, reduces the rate of misjudgment, and provides more stable attribute judgment results in the vehicle cockpit environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027918A_ABST
    Figure CN119027918A_ABST
Patent Text Reader

Abstract

The invention relates to an in-vehicle personnel attribute determination method and device, a vehicle-mounted terminal and a storage medium, and the method comprises the steps: obtaining the video data of a vehicle cabin, performing human body object detection on each video image in the video data by using a human body detection network model to obtain a human body external frame detection result and an attribute detection result corresponding to a human body object in each video image; sequentially matching the human body external frame detection results of the two adjacent frames of video images according to the video image time sequence, and summarizing the attribute detection results of the two successfully matched adjacent frames of video images to the same human body object; and voting the attribute detection result of each human body object to obtain a target attribute detection result corresponding to each human body object. According to the method, the time sequence information of the video images is introduced, the attribute detection result of each video image in the video data is concluded and voted, which is equivalent to smoothing the attribute detection result based on the duration, so that the accuracy of the attribute detection result is greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method and device for determining attributes of a person in a vehicle, a vehicle-mounted terminal, and a readable storage medium. Background Art

[0002] With the development of intelligent cars, car users have also put forward higher requirements for safety. Identifying whether the occupants are adults or children is one of the important means to improve the safety of the vehicle cabin.

[0003] At present, the human key point detection network is usually used to identify the input human image to obtain the geometric position information of the human key points, and then the length information or proportion information such as shoulder width and torso is calculated through the human key points to distinguish whether it is an adult or a child. When judging human attributes through geometric information, it is necessary to rely on experience and artificially set various complex logics to achieve relatively accurate results. Such a design is difficult and not very robust. In a complex environment such as a cockpit, there are many occlusion and interference factors. For example, if the human target behaves violently or suddenly enters an occlusion state, the human key point detection result will be inaccurate, which will affect the accuracy of the human attribute judgment result. In addition to using the human key point detection network to realize human attribute detection, the human detection classification network can also be used to identify the input human image and directly obtain the attribute detection result as an adult or a child. However, in the complex environment inside the vehicle cockpit, the accuracy of distinguishing human attributes only through the human detection classification network is not high, and it is highly dependent on training data. When some uncommon scenes appear inside the vehicle cockpit, the human detection classification network may make incorrect judgments due to the lack of corresponding training data in the database involved in the training.

[0004] Therefore, how to improve the accuracy of attribute detection results is an urgent problem to be solved. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a method, device, vehicle-mounted terminal and storage medium for determining attributes of a person in a vehicle.

[0006] In a first aspect, the present disclosure provides a method for determining attributes of a person in a vehicle, comprising:

[0007] Acquire video data of a vehicle cabin, and use a human body detection network model to perform human body object detection on video images in the video data to obtain human body external frame detection results and attribute detection results corresponding to human body objects in each of the video images; the human body external frame detection results include the positions of the detected human body objects in the video images; the attribute detection results are used to indicate whether the detected human body objects are adults or children;

[0008] Sequentially matching the human body external frame detection results of two adjacent frames of the video images according to the time sequence of the video images, so as to classify the attribute detection results corresponding to the two adjacent frames of the video images that have been successfully matched into the same human body object;

[0009] The attribute detection results corresponding to each of the human objects are voted respectively to obtain the target attribute detection results corresponding to each of the human objects.

[0010] In some embodiments, sequentially matching the human body external frame detection results of two adjacent frames of the video images according to the time sequence of the video images to classify the attribute detection results corresponding to the two adjacent frames of the video images that have been successfully matched as the same human body object includes:

[0011] Matching the detection results of the human body external frame corresponding to two adjacent frames of the video images;

[0012] If the match is successful, the attribute detection result of the video image in the latter frame of the two adjacent frames is stored in the human object identifier corresponding to the attribute detection result of the successful match; if the match fails, a new human object identifier is created, and the human attribute result of the video image in the latter frame of the two adjacent frames is stored in the newly created human object identifier;

[0013] Matching is performed from right to left in the time sequence of the video images until the human object to which the human attribute detection result of the last frame of the video image belongs is determined.

[0014] In some embodiments, the step of sequentially matching the detection results of the human body external frame corresponding to two adjacent frames of the video images according to the time sequence of the video images includes:

[0015] According to the detection result of the human body frame of the video image of the previous frame in two adjacent frames, the prediction result of the human body frame corresponding to the video image of the next frame is calculated by the Kalman filter matching algorithm;

[0016] The human body external frame detection results corresponding to the video image of the next frame are matched with the human body external frame prediction results.

[0017] In some embodiments, before voting on the attribute detection results corresponding to the human objects to obtain the target attribute detection results corresponding to the human objects, the method further includes:

[0018] The attribute detection results corresponding to each of the human objects are voted respectively to obtain the target attribute detection results corresponding to each of the human objects.

[0019] In some embodiments, voting on the attribute detection results corresponding to the human objects to obtain the target attribute detection results corresponding to the human objects includes:

[0020] For any valid human subject, counting the number of times the attribute detection result is an adult and the number of times the attribute detection result is a child;

[0021] If it is determined that the number of times the attribute detection result is an adult is greater than the number of times the attribute detection result is a child, then it is determined that the target attribute detection result corresponding to the valid human object is an adult;

[0022] If it is determined that the number of times the attribute detection result is a child is less than or equal to the number of times the attribute detection result is an adult, it is determined that the target attribute detection result corresponding to the valid human object is a child.

[0023] In some embodiments, the method further comprises:

[0024] Detecting body movements of a human subject in the video data;

[0025] An early warning operation is performed according to the body movements of the human object and the target attribute detection result corresponding to the human object.

[0026] In a second aspect, the present disclosure provides a device for determining attributes of a person in a vehicle, comprising:

[0027] An acquisition module, used for acquiring video data of a vehicle cockpit;

[0028] A detection module, configured to perform human object detection on each video image in the video data using a human detection network model to obtain a human body bounding box detection result and an attribute detection result corresponding to the human body object in each video image; the human body bounding box detection result includes a position of the detected human body object in the video image; and the attribute detection result is used to indicate whether the detected human body object is an adult or a child;

[0029] A matching module, used for sequentially matching the detection results of the human body external frame of two adjacent frames of the video images according to the time sequence of the video images, so as to classify the attribute detection results corresponding to the two adjacent frames of the video images that have been successfully matched into the same human body object;

[0030] The voting processing module is used to vote on the attribute detection results corresponding to each of the human objects to obtain the target attribute detection results corresponding to each of the human objects.

[0031] In a third aspect, the present disclosure provides a vehicle-mounted terminal, including: a memory and a processor;

[0032] The memory is configured to store computer program instructions; the processor is configured to execute the computer program instructions, so that the vehicle-mounted terminal implements the method for determining the attributes of the person in the vehicle as described in the first aspect and any one of the first aspects.

[0033] In a fourth aspect, the present disclosure provides a readable storage medium, comprising: computer program instructions; at least one processor of a vehicle-mounted terminal runs the computer program instructions, so that the vehicle-mounted terminal implements the method for determining the attributes of a person in a vehicle as described in the first aspect and any one of the first aspects.

[0034] In a fifth aspect, the present disclosure provides a computer program product, and a vehicle-mounted terminal runs the computer program product, so that the vehicle-mounted terminal implements the method for determining attributes of a person in a vehicle as described in the first aspect and any one of the first aspects.

[0035] In a sixth aspect, the present disclosure provides a vehicle, comprising one or more of the device for determining attributes of persons in a vehicle as described in the second aspect, the vehicle-mounted terminal as described in the third aspect, and the readable storage medium as described in the fourth aspect.

[0036] The disclosed embodiment provides a method, device, vehicle-mounted terminal and storage medium for determining the attributes of people in a vehicle, wherein the method comprises: obtaining video data of a vehicle cabin, and using a human body detection network model to perform human body object detection on each video image in the video data to obtain a human body external frame detection result and an attribute detection result corresponding to the human body object in each video image; the human body external frame detection result includes the position of the detected human body object in the video image; the attribute detection result is used to indicate that the detected human body object is an adult or a child; the human body external frame detection results of two adjacent frames of video images are matched in sequence according to the time sequence of the video images, so as to summarize the attribute detection results corresponding to the two adjacent frames of video images that are successfully matched to the same human body object; and the attribute detection results corresponding to each human body object are voted respectively to obtain the target attribute detection results corresponding to each human body object. The disclosed method summarizes and votes the attribute detection results of each video image in the video data by introducing the video image time sequence information, which is equivalent to performing long-term smoothing on the attribute detection results, thereby greatly improving the accuracy of the attribute detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0039] Figure 1 A schematic diagram of a flow chart of a method for determining attributes of a person in a vehicle provided by an embodiment of the present disclosure;

[0040] Figure 2 A schematic diagram of a human body detection network model provided by an embodiment of the present disclosure;

[0041] Figure 3 A schematic diagram of a flow chart of a method for determining attributes of a person in a vehicle provided by another embodiment of the present disclosure;

[0042] Figure 4 A schematic diagram of a flow chart of a method for determining attributes of a person in a vehicle provided by another embodiment of the present disclosure;

[0043] Figure 5 A structural diagram of a device for determining attributes of a person in a vehicle provided by an embodiment of the present disclosure;

[0044] Figure 6 A structural diagram of a vehicle-mounted terminal provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0045] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0046] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete implementation method separately. It should be noted that the brief description of the terms in this application is only for the convenience of understanding the implementation methods described below, and is not intended to limit the implementation methods of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0047] The above two traditional methods both obtain attribute detection results based on a single captured cockpit image, and do not take into account the temporal continuity of the cockpit space. Therefore, the present disclosure provides a method for determining the attributes of people in a vehicle. By processing a continuous and stable video stream and utilizing the strong correlation between frames, the attribute detection results within a time period are summarized and voted, which is equivalent to smoothing the attribute detection results based on the duration, thereby improving the accuracy of human attribute detection.

[0048] The method provided by the present disclosure can be implemented by the device for determining the attributes of people in the vehicle provided by the present disclosure. The device can be implemented by any software and / or hardware method. The device for determining the attributes of people in the vehicle can be configured in a vehicle, which may be but is not limited to any of the following types of vehicles: smart cars, new energy vehicles, ordinary passenger cars, convertible passenger cars, high-end passenger cars, small passenger cars, convertibles, hatchback passenger cars, station wagons, multi-purpose passenger cars, short-head passenger cars, off-road passenger cars, special-purpose passenger cars, etc.

[0049] Next, the method for determining attributes of occupants in a vehicle provided by the present disclosure will be described in detail in combination with the accompanying drawings and application scenarios.

[0050] Figure 1 This is a flow chart of a method for determining attributes of a person in a vehicle provided by an embodiment of the present disclosure. Figure 1 As shown, the method provided in this embodiment may include:

[0051] S101, obtaining video data of a vehicle cabin.

[0052] The vehicle cabin refers to the interior space of the vehicle, which is a compartment for the driver and passengers. The vehicle cabin is photographed using an image sensor (such as a camera) to obtain video data of the vehicle cabin. The image sensor can be set on the vehicle. Some types of vehicles may include multiple compartments, and one or more image sensors can be set for each compartment. In addition, the present disclosure does not limit the parameters such as the model, pixel, and installation location of the image sensor.

[0053] Taking the image sensor as a camera as an example, during the driving of the vehicle, the camera can capture video data of a preset time, which is not limited in the present disclosure, such as 5s, 8s, 10s, etc. The image sensor then sends the captured video data to the in-vehicle occupant attribute determination device.

[0054] S102, using a human body detection network model to perform human body object detection on video images in the video data to obtain human body bounding box detection results and attribute detection results corresponding to human body objects in each video image.

[0055] The trained human body detection network model is pre-deployed in the vehicle occupant attribute determination device. The human body detection network model is a deep learning network built and trained for human body attribute detection and classification scenarios. For example, see Figure 2As shown, the human body detection network model may include a backbone network, a shoulder network, and a head network. The human body detection network model can detect whether there is a human body object in the input image and return the human body bounding box coordinates (i.e., the human body bounding box detection result) and human body attributes (i.e., the attribute detection result) corresponding to the human body object. Before the image is input into the human body detection network model, the image can be preprocessed to convert the image from pixel data to vector expression so that the human body detection network model can process it. Similarly, the human body bounding box detection results and attribute detection results output by the human body detection network model also need to be post-processed and converted into expressions that can be understood by the user.

[0056] The video data is used as input, and the human body detection network model is used to identify each frame of the video data to obtain the human body bounding box detection result and attribute detection result of each frame of the video image. Among them, the human body bounding box detection result is the position of the detected human object in the corresponding video image, and the position can be represented by image coordinates. The attribute detection result is used to indicate whether the detected human object is an adult or a child.

[0057] It should be noted that since human objects in the vehicle cabin may perform violent behaviors or be blocked due to movement, human objects may not exist in some video images. Therefore, the human body external frame detection results and attribute detection results corresponding to such video images are both no results.

[0058] S103, sequentially matching the human body external frame detection results of two adjacent frames of the video images according to the time sequence of the video images, so as to classify the attribute detection results corresponding to the two adjacent frames of the video images that are successfully matched into the same human body object.

[0059] The device for determining the attributes of people in the vehicle matches the human body external frame detection results corresponding to two adjacent frames of video images in sequence according to the video image time sequence, determines whether the human body objects detected in the two adjacent frames of video images are the same human body object, and executes it in sequence from front to back in the video image time sequence, and then summarizes and counts the attribute detection results of different video images in the entire video data for the same human body object. Through matching and summarization, one or more attribute detection results corresponding to each detected human body object can be obtained.

[0060] The present disclosure may, but is not limited to, match the human body external frame detection results of each video image through Kalman filtering, limiting filtering, median filtering, mean filtering, one-Euro filtering, etc.

[0061] Among them, the device for determining the attributes of people in the vehicle can use multi-threading to perform human object detection and the matching and induction of human body external frame detection results in parallel, which is conducive to improving processing efficiency.

[0062] S104, voting on the attribute detection results corresponding to the human body objects respectively to obtain the target attribute detection results corresponding to the human body objects.

[0063] For each human object, the attribute detection results are counted to obtain the number of times a when the attribute detection result is an adult and the number of times b when the attribute detection result is a child. Then, by comparing the two, if a is greater than b, it is determined that the target attribute detection result of the human object is an adult. If a is less than or equal to b, it is determined that the target attribute detection result of the human object is a child.

[0064] When voting, you can vote for all the attribute detection results of a human object, or you can vote for some of the attribute detection results. For example, when the overall duration of the video data is long, you can take a later video sub-segment and vote on the attribute detection results of the human object detected in the video image within the video sub-segment, where the duration of the video sub-segment can be obtained by statistics based on historical voting processing results, thereby ensuring the accuracy of the target attribute detection results obtained by voting. Of course, you can also vote for all the attribute detection results of the human object. The present disclosure does not limit the voting method.

[0065] The method of this embodiment performs smoothing processing on the attribute detection results of multiple frames of video images within a period of time based on the timing information of the video images, and the obtained results are very stable within a period of time. Compared with the traditional method of determining human attributes based on single-frame images, the accuracy of human attribute detection results is greatly improved, and there will be no repeated jump phenomenon. In addition, the method of this embodiment can also greatly reduce the dependence of the human body detection network model on training data. Since most passengers in the vehicle cabin are in a normal posture under the driving environment, the video data collected in the normal posture can be used as sample data for training, and then the erroneous results in the abnormal posture can be corrected by the timing information of the video image, thereby solving the large amount of abnormal posture data collection requirements of the human body detection network model (abnormal postures are of various types, difficult to collect and large data requirements).

[0066] In a specific embodiment, a Kalman filter algorithm is used to match the detection results of the human body external frame of the video image and summarize the attribute detection results. Kalman filtering is an efficient recursive filter (autoregressive filter) that can estimate the state of a dynamic system from a series of incomplete and noisy measurements. In the present disclosure, the coordinates of the human body external frame of the human object can be estimated by using Kalman filtering. Among them, the recognition and matching processing of each frame of the video image according to the video image time sequence using the human body detection network model can be performed in parallel, and parallel execution can greatly improve the processing efficiency.

[0067] Please refer to Figure 3 As shown, the current frame video image is input into the human body detection network model to obtain the human body external frame detection result and attribute detection result; next, the human body external frame detection result and human body object identification of the previous frame video image are called, and the human body external frame detection result of the current frame video image and the human body external frame detection result of the previous frame video image are matched by Kalman filtering; if the match is successful, the attribute detection result corresponding to the human body object successfully matched in the current frame video image is stored in the human body object identification corresponding to the human body object successfully matched in the previous frame video image; if the match fails, a new human body object identification is created, and the attribute detection result corresponding to the human body object that failed to match in the current frame video image is stored in the new human body object identification. Matching is performed according to the video image timing until the matching of the last frame video image of the video data is completed, and then the attribute detection result corresponding to all human body objects is obtained by voting to obtain the attribute detection result after smoothing based on the video image timing information.

[0068] Exemplarily, when matching two adjacent frames of video images, for the previous frame of video image, it is assumed that a human object is detected, and its human object identifier (such as ID) is recorded as ID-1, and ID-1 saves all the attribute detection results of the ID in the previous frame. For the next frame of video image, it is assumed that two human objects are detected. For the human object corresponding to ID-1, the human body bounding box prediction result of ID-1 in the next frame of video image can be obtained by the Kalman filter algorithm, and then the human body bounding box prediction result of ID-1 in the next frame of video image is matched with the human body bounding box detection result output by the human detection network model for the next frame of video image. As previously assumed, two human objects are detected in the next frame of video image, and the human body bounding box prediction result of ID-1 can be matched with the human body bounding box detection results corresponding to the two human objects to be matched in the next frame of video image to obtain a matching result. If there is a successfully matched human object, the attribute detection result corresponding to the human object that successfully matches ID-1 in the next frame of video image is stored in ID-1; for the human object that is not successfully matched, a new human object identifier is created, such as ID-2, and the attribute detection result corresponding to the human object that is not successfully matched is stored in the newly created ID-2.

[0069] Among them, by matching the human body external bounding box detection result and the human body external bounding box prediction result, the distance between the coordinates can be calculated. If the distance is less than the preset distance, the match is determined to be successful. If the distance is greater than or equal to the preset distance, the match fails.

[0070] Considering that there may be invalid human objects, for example, some people who are not in the cabin appear in the cabin and stay for a short time, such people can be regarded as invalid human objects and no attribute detection is required. Figure 1 as well as Figure 3 Based on the illustrated embodiment, invalid human objects can be eliminated before voting, and the attribute detection results of the retained valid human objects can be voted on to reduce interference factors and improve the accuracy of the detection results, and the amount of calculation can be reduced to improve processing efficiency.

[0071] In some embodiments, whether the detected human object is an invalid human object can be determined based on the time that the detected human object appears continuously in the video data. Usually, the video data is shot at a certain video frame rate, and the time can also be converted into the number of video images. If it is determined that a human object is not detected in a preset number of video images continuously, the human object is determined to be an invalid human object. The value of the preset number can be obtained by multiplying the pre-set time length for determining an invalid human object and the shooting video frame rate, or it can also be a fixed value.

[0072] Figure 4 This is a flow chart of a method for determining attributes of a person in a vehicle provided by another embodiment of the present disclosure. Figure 4 As shown, the method of this embodiment includes:

[0073] S401: Detect body movements of a human subject in the video data.

[0074] On the basis of eliminating invalid human objects, the in-vehicle occupant attribute determination device can detect the body movements of the retained valid human objects. In some embodiments, the outline of the human object and the key points of the body in the outline can be identified through the human body external frame detection result, and then the user's body movements can be determined based on the key points of the body in the outline.

[0075] S402: Perform a warning operation according to the body movements of the human subject and the target human attribute detection result corresponding to the human subject.

[0076] The in-vehicle occupant attribute determination device may pre-store the mapping relationship between adults, body movements, and warning operations, and the mapping relationship between children, body movements, and warning operations. Figures 1 to 3 After the method of the illustrated embodiment determines the target attribute detection result of the human object, it queries the matching mapping relationship. If there is a matching item in the mapping relationship, the warning operation is determined according to the matching item and the warning operation is performed; if there is no matching item in the mapping relationship, it ends without performing any operation. The present disclosure does not limit the warning operation, and different body movements can correspond to different warning operations. The warning operation can include but is not limited to: voice reminder operation, displaying preset text content on the display screen of the vehicle, etc.

[0077] In addition, the cabin section can be determined according to the position of the camera collecting video data in the vehicle, and the seat position of the human object in the cabin section can be determined according to the video data. When performing the warning operation, the cabin section information and seat information can be output, so that the warning operation is more targeted and the passengers who may be in danger can be quickly and timely reminded. In addition, because the human attributes determined by the above method are more accurate, the probability of false triggering of warnings can be reduced, and the user experience can be improved.

[0078] Figure 5 This is a structural diagram of a device for determining attributes of a person in a vehicle provided by an embodiment of the present disclosure. Figure 5 As shown, the device 500 provided in this embodiment includes:

[0079] An acquisition module 501 is used to acquire video data of a vehicle cabin;

[0080] The detection module 502 is used to perform human object detection on the video images in the video data using a human detection network model to obtain a human body bounding box detection result and an attribute detection result corresponding to the human body object in each of the video images; the human body bounding box detection result includes a position of the detected human body object in the video image; and the attribute detection result is used to indicate whether the detected human body object is an adult or a child;

[0081] The matching module 503 is used to sequentially match the human body external frame detection results of two adjacent frames of the video image according to the time sequence of the video image, so as to classify the attribute detection results corresponding to the two adjacent frames of the video image that are successfully matched into the same human body object;

[0082] The voting processing module 504 is used to vote on the attribute detection results corresponding to each of the human objects to obtain the target attribute detection results corresponding to each of the human objects.

[0083] In some embodiments, the matching module 503 is specifically used to match the human body external frame detection results corresponding to the video images of two adjacent frames; if the match is successful, the attribute detection result of the video image of the latter frame between the two adjacent frames is stored in the human body object identifier corresponding to the successfully matched attribute detection result; if the match fails, a new human body object identifier is created, and the human body attribute result of the video image of the latter frame between the two adjacent frames is stored in the newly created human body object identifier; matching is performed from front to back according to the time sequence of the video images until the human body object to which the human body attribute detection result of the last frame of the video image belongs is determined.

[0084] In some embodiments, the matching module 503 is specifically used to calculate the human body external frame prediction result corresponding to the video image of the next frame through the Kalman filter matching algorithm based on the human body external frame detection result of the video image of the previous frame in two adjacent frames; and match the human body external frame detection result corresponding to the video image of the next frame with the human body external frame prediction result.

[0085] In some embodiments, the voting processing module 504 is further used to remove invalid human objects from all human object identifiers before voting on the attribute detection results corresponding to each of the human objects to obtain the target attribute detection results corresponding to each of the human objects. Accordingly, the voting processing module 504 is specifically used to vote on the attribute detection results corresponding to the valid human objects that have not been removed to obtain the target attribute detection results corresponding to the valid human objects.

[0086] In some embodiments, the voting processing module 504 is specifically configured to determine that any human object is an invalid human object and eliminate the invalid human object if it is analyzed that any human object does not appear in a preset number of video images continuously.

[0087] In some embodiments, the voting processing module 504 is specifically used to count the number of times the attribute detection result is an adult and the number of times the attribute detection result is a child for any valid human object; if the number of times the attribute detection result is an adult is greater than the number of times the attribute detection result is a child, then it is determined that the target attribute detection result corresponding to the valid human object is an adult; if the number of times the attribute detection result is a child is less than or equal to the number of times the attribute detection result is an adult, then it is determined that the target attribute detection result corresponding to the valid human object is a child.

[0088] In some embodiments, the device 500 further includes: an early warning module 505, which is used to detect the body movements of the human object in the video data; and perform early warning operations according to the body movements of the human object and the target attribute detection results corresponding to the human object.

[0089] The device provided in this embodiment can be used to implement the technical solution of any of the aforementioned method embodiments. Its implementation principle and technical effects are similar. Please refer to the detailed description of the aforementioned method embodiments. For the sake of brevity, they will not be repeated here.

[0090] Figure 6 This is a structural diagram of a vehicle-mounted terminal provided by an embodiment of the present disclosure. Figure 6 As shown, the vehicle-mounted terminal 600 provided in this embodiment includes: a memory 601 and a processor 602 .

[0091] The memory 601 may be an independent physical unit, and may be connected to the processor 602 via a bus 603. The memory 601 and the processor 602 may also be integrated together and implemented by hardware. The memory 601 is used to store program instructions, and the processor 602 calls the program instructions to execute the method for determining the attributes of the person in the vehicle provided by any of the above method embodiments.

[0092] Optionally, when part or all of the methods of the above embodiments are implemented by software, the vehicle-mounted terminal 600 may also include only a processor 602. The memory 601 for storing programs is located outside the vehicle-mounted terminal 600, and the processor 602 is connected to the memory through circuits / wires to read and execute the programs stored in the memory. The processor 602 may be a central processing unit (CPU), a network processor (NP) or a combination of a CPU and a NP. The processor 602 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0093] The memory 601 may include a volatile memory, such as a random-access memory (RAM); the memory may also include a non-volatile memory, such as a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory may also include a combination of the above types of memory.

[0094] Exemplarily, the present disclosure provides a chip system, which is applied to a vehicle-mounted terminal including a memory and a sensor; the chip system includes: a processor; the processor is used to execute the method for determining the attributes of the person in the vehicle of the above embodiment.

[0095] Exemplarily, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor of a vehicle-mounted terminal so that the vehicle-mounted terminal implements the method for determining attributes of a person in a vehicle of the foregoing embodiment when executing the computer program.

[0096] Exemplarily, the present disclosure provides a computer program product, which, when executed on a vehicle-mounted terminal, enables the vehicle-mounted terminal to execute the method for determining attributes of a person in a vehicle of the foregoing embodiment.

[0097] Exemplarily, the present disclosure provides a vehicle, comprising one or more of the device for determining attributes of a person in the vehicle as described in the second aspect, the vehicle-mounted terminal as described in the third aspect, and the readable storage medium as described in the fourth aspect.

[0098] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0099] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining attributes of people in a vehicle, characterized in that: include: Acquire video data of a vehicle cabin, and use a human body detection network model to perform human body object detection on each video image in the video data to obtain a human body external frame detection result and an attribute detection result corresponding to the human body object in each video image; the human body external frame detection result includes the position of the detected human body object in the video image; the attribute detection result is used to indicate whether the detected human body object is an adult or a child; Sequentially matching the human body external frame detection results of two adjacent frames of the video images according to the time sequence of the video images, so as to classify the attribute detection results corresponding to the two adjacent frames of the video images that have been successfully matched into the same human body object; The attribute detection results corresponding to each of the human objects are voted respectively to obtain the target attribute detection results corresponding to each of the human objects.

2. The method according to claim 1, characterized in that The method of sequentially matching the human body external frame detection results of two adjacent frames of the video images according to the time sequence of the video images, so as to classify the attribute detection results corresponding to the two adjacent frames of the video images that have been successfully matched into the same human body object, includes: Matching the detection results of the human body external frame corresponding to two adjacent frames of the video images; If the match is successful, the attribute detection result of the video image in the latter frame of the two adjacent frames is stored in the human object identifier corresponding to the attribute detection result of the successful match; if the match fails, a new human object identifier is created, and the human attribute result of the video image in the latter frame of the two adjacent frames is stored in the newly created human object identifier; Matching is performed from front to back according to the time sequence of the video images until the human object to which the human attribute detection result of the last frame of the video image belongs is determined.

3. The method according to claim 1, characterized in that The step of sequentially matching the detection results of the human body external frames corresponding to two adjacent frames of the video images according to the time sequence of the video images includes: According to the detection result of the human body frame of the video image of the previous frame in two adjacent frames, the prediction result of the human body frame corresponding to the video image of the next frame is calculated by the Kalman filter matching algorithm; The human body external frame detection results corresponding to the video image of the next frame are matched with the human body external frame prediction results.

4. The method according to claim 1, characterized in that: Before voting on the attribute detection results corresponding to the human objects to obtain the target attribute detection results corresponding to the human objects, the method further includes: If it is analyzed that any of the human body objects does not appear in a preset number of video images continuously, the human body object is determined to be an invalid human body object and is removed.

5. The method according to claim 4, characterized in that The method of voting on the attribute detection results corresponding to the human body objects to obtain the target attribute detection results corresponding to the human body objects includes: For any valid human subject, counting the number of times the attribute detection result is an adult and the number of times the attribute detection result is a child; If it is determined that the number of times the attribute detection result is an adult is greater than the number of times the attribute detection result is a child, then it is determined that the target attribute detection result corresponding to the valid human object is an adult; If it is determined that the number of times the attribute detection result is a child is less than or equal to the number of times the attribute detection result is an adult, it is determined that the target attribute detection result corresponding to the valid human object is a child.

6. The method according to claim 1, characterized in that The method further comprises: Detecting body movements of a human subject in the video data; An early warning operation is performed according to the body movements of the human object and the target attribute detection result corresponding to the human object.

7. A device for determining attributes of people in a vehicle, characterized in that: include: An acquisition module, used for acquiring video data of a vehicle cockpit; A detection module, configured to perform human object detection on each video image in the video data using a human detection network model to obtain a human body bounding box detection result and an attribute detection result corresponding to the human body object in each video image; the human body bounding box detection result includes a position of the detected human body object in the video image; and the attribute detection result is used to indicate whether the detected human body object is an adult or a child; A matching module, used for sequentially matching the detection results of the human body external frame of two adjacent frames of the video images according to the time sequence of the video images, so as to classify the attribute detection results corresponding to the two adjacent frames of the video images that have been successfully matched into the same human body object; The voting processing module is used to vote on the attribute detection results corresponding to each of the human objects to obtain the target attribute detection results corresponding to each of the human objects.

8. A vehicle-mounted terminal, characterized in that: include: Memory and processor; The memory is configured to store computer program instructions; The processor is configured to execute the computer program instructions so that the vehicle-mounted terminal implements the method for determining attributes of a person in a vehicle as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that: include: Computer program instructions; At least one processor of the vehicle-mounted terminal runs the computer program instructions, so that the vehicle-mounted terminal implements the method for determining the attributes of the person in the vehicle as described in any one of claims 1 to 6.

10. A vehicle, characterized in that: include: The device for determining attributes of a person in a vehicle as claimed in claim 7; or, The vehicle-mounted terminal as claimed in claim 8; or, The readable storage medium as claimed in claim 9.