Target object detection method, device, vehicle and storage medium

Abstract

The embodiments of the present application disclose a method, device, vehicle and storage medium for detecting a target object. The method comprises: obtaining an image of the interior of the vehicle collected in real time by a collection device, and inputting the image into a pre-constructed detection model, wherein the detection model is constructed based on meta-learning; obtaining the target object output by the detection model after image recognition of the image; if there is a dangerous target object among the targets, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle. Through the above method, the pre-constructed meta-learning-based detection model can be used to identify targets in the vehicle with high precision, thereby improving the recognition rate of the targets. In addition, when a dangerous target object is detected, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle, thereby reducing the occurrence of traffic accidents.

Description

Technical Field

[0001] The present application relates to the field of intelligent vehicle technology, and more specifically, to a target object detection method, device, vehicle and storage medium. Background Art

[0002] With the development of computer vision technology and the popularization of autonomous driving and in-vehicle intelligent application scenarios, there is an increasing demand for vehicles to use cameras for target recognition, such as in-vehicle occupant recognition, driver eye tracking and expression recognition, etc. In related target detection methods, the recognition rate of targets needs to be improved. Summary of the invention

[0003] In view of the above problems, the present application proposes a target detection method, device, vehicle and storage medium to improve the above problems.

[0004] In a first aspect, the present application provides a method for detecting a target object, the method comprising: obtaining an image of the interior of a vehicle collected in real time by an acquisition device, and inputting the image into a pre-constructed detection model, wherein the detection model is constructed in a meta-learning manner; obtaining a target object output by the detection model after performing image recognition on the image; if there is a dangerous target object among the targets, triggering an alarm to prompt the driver to pay attention to the condition inside the vehicle.

[0005] In the second aspect, the present application provides a target object detection device, which includes: an image input unit, used to obtain the image of the interior of the vehicle collected in real time by an acquisition device, and input the image into a pre-constructed detection model, wherein the detection model is constructed based on meta-learning; a target object acquisition unit, used to obtain the target object output by the detection model after image recognition of the image; an alarm unit, used to trigger an alarm if there is a dangerous target among the targets, so as to prompt the driver to pay attention to the situation inside the vehicle.

[0006] In a third aspect, the present application provides a vehicle comprising one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0007] In a fourth aspect, the present application provides a computer-readable storage medium, in which program code is stored, and the program code can be called by a processor to execute the above method.

[0008] The present application provides a method, device, vehicle and storage medium for detecting a target object. First, an image of the interior of the vehicle collected in real time by a collection device is obtained, and the image is input into a pre-built detection model, which is built based on meta-learning. Then, the target object output by the detection model after image recognition is obtained. If there is a dangerous target among the targets, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle. Through the above method, the pre-built meta-learning-based detection model can be used to identify the targets in the vehicle with high precision, thereby improving the recognition rate of the targets. In addition, when a dangerous target is detected, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle, which can reduce the occurrence of traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flow chart of a target object detection method proposed in the present application is shown;

[0011] Figure 2 A schematic diagram showing the division of a grid ruler in a target object detection method proposed in the present application is shown;

[0012] Figure 3 A flow chart showing another target object detection method proposed in the present application is shown;

[0013] Figure 4 A flow chart of another target object detection method proposed in the present application is shown;

[0014] Figure 5 A flow chart of another target object detection method proposed in the present application is shown;

[0015] Figure 6 A structural block diagram of a target object detection device proposed in the present application is shown;

[0016] Figure 7 A structural block diagram of another target detection device proposed in the present application is shown;

[0017] Figure 8 A structural block diagram of a vehicle for executing a target object detection method according to an embodiment of the present application is shown;

[0018] Fig. 9It is a storage medium for storing or carrying program codes for implementing the target object detection method according to the embodiment of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] With the development of computer vision technology and the popularization of autonomous driving and in-vehicle intelligent application scenarios, there is an increasing demand for vehicles to use cameras to identify targets, such as: in-vehicle occupant identification, driver eye tracking and expression recognition, etc.

[0021] However, the inventors found in their research on related target object detection methods that the recognition rate of the target objects by the related target object detection methods needs to be improved.

[0022] Therefore, the inventor proposed the method, device, vehicle and storage medium in this application, which first obtains the image of the interior of the vehicle collected in real time by the collection device, and inputs the image into a pre-constructed detection model, wherein the detection model is constructed based on meta-learning, and then obtains the target object output after the detection model performs image recognition on the image. If there is a dangerous target among the targets, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle. The pre-constructed meta-learning-based detection model can be used to identify the targets in the vehicle with high precision, thereby improving the recognition rate of the targets. In addition, when a dangerous target is detected, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle, thereby reducing the occurrence of traffic accidents caused by distracted driver attention.

[0023] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0024] See also Figure 1 The present application provides a target object detection method, which is applied to a vehicle, and the method comprises:

[0025] Step 110: Obtain the image of the interior of the vehicle collected by the collection device in real time, and input the image into a pre-built detection model, where the detection model is built based on meta-learning.

[0026] The acquisition device may be a vehicle-mounted camera installed inside the vehicle. Optionally, the acquisition device may also be a smart phone and a driving recorder.

[0027] As a method, the pre-built detection model can be in a server or in a vehicle-mounted system, wherein the vehicle-mounted system can be a system with data processing functions installed inside the vehicle. Furthermore, when the image of the interior of the vehicle is captured in real time by a vehicle-mounted camera installed inside the vehicle, the captured image can be sent to the server or vehicle-mounted system where the pre-built detection model is located.

[0028] Optionally, when it is detected that the vehicle starts to start, the acquisition device starts to acquire images of the interior of the vehicle.

[0029] Specifically, when it is detected that the vehicle's engine is started, the acquisition device begins to acquire images of the interior of the vehicle in real time, and stores the real-time acquired images of the interior of the vehicle in a storage area of ​​the vehicle. When it is necessary to obtain the images acquired by the acquisition device, the images of the interior of the vehicle acquired by the acquisition device can be read from the storage area.

[0030] Step 120: Obtain the target object output by the detection model after performing image recognition on the image.

[0031] The target object may be understood as an object to be identified, and optionally, the target object may be a person or other object. As a method, the target object may be identified by the detection model according to feature information included in the extracted input image.

[0032] Optionally, when the image of the interior of the vehicle is input into the detection model, the detection model divides the input image of the interior of the vehicle into coarse, medium and fine grids, so as to respectively predict large-sized objects, medium-sized objects and small-sized objects in the image. For example, if the size of the input image is 416*416, the sizes of the coarse, medium and fine grids can be 13*13, 26*26 and 52*52 respectively. Figure 2 shown.

[0033] Specifically, the input image can be divided into S*S grids, each grid generates B bounding boxes, and each bounding box predicts 6 values, namely: position information (x, y, w, h), confidence, and angle. Among them, (x, y) represents the coordinates of the bounding box in the grid, (w, h) is the height and width of the bounding box relative to the entire image, and each bounding box predicts the category information of an object according to the category confidence.

[0034] Furthermore, the detection model can output each target object included in the image in the above manner. Then, the vehicle can obtain the target object output by the detection model after performing image recognition on the image from the server.

[0035] Step 130: If there is a dangerous object among the objects, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle.

[0036] As a method, when the vehicle obtains that the detection model outputs the target object in the image as a dangerous target based on the feature information extracted from the image, an alarm is triggered, wherein the alarm method may include voice prompts, text reminders, and light reminders.

[0037] Specifically, the alarm can be issued by voice prompting, which can be sending a trigger instruction to an FM player or a music player, so that the FM player or the music player broadcasts the current status inside the vehicle. The trigger instruction can be turning on the FM player or the music player and playing the corresponding audio. Furthermore, the corresponding audio can be a reminder audio pre-stored in the FM player or the music player.

[0038] The present embodiment provides a method for detecting a target object. First, an image of the interior of the vehicle collected in real time by a collection device is obtained, and the image is input into a pre-built detection model, which is built based on meta-learning. Then, the target object output by the detection model after image recognition is obtained. If there is a dangerous target among the targets, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle. Through the above method, the pre-built meta-learning-based detection model can be used to identify the targets in the vehicle with high precision, thereby improving the recognition rate of the targets. In addition, when a dangerous target is detected, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle, which can reduce the occurrence of traffic accidents.

[0039] See also Figure 3 The present application provides a target object detection method, which is applied to a vehicle, and the method comprises:

[0040] Step 210: Build a detection model.

[0041] As a method, the step of constructing the detection model includes: obtaining a specified number of images of the interior of the vehicle collected in advance by the collection device; calibrating the images, dividing the images with the same semantics in the images into one category, and obtaining calibrated images; dividing the calibrated images into training set images, test set images, and verification set images in proportion, wherein the types of images included in the training set images, the test set images, and the verification set images are different; inputting the training set images into the initial detection model to train the initial detection model; inputting the test set images and the verification set images into the initial detection model to train the initial detection model; to the trained initial detection model to obtain a loss function of the target object recognition rate, and adjust the model parameters of the trained initial detection model multiple times based on the loss function to obtain the detection model with the highest target object recognition rate, and use the detection model with the highest target object recognition rate as the pre-built detection model, wherein the expression of the loss function is loss=coordErr+iouErr+clsErr, wherein loss represents the loss function, coordErr represents the coordinate loss function, iouErr represents the confidence loss function, and claErr represents the category loss function.

[0042] Optionally, the specified number of vehicle interior images may be small sample images collected, wherein the small sample images may be a sample size of the collected images that is less than or equal to 30.

[0043] Optionally, a calibration tool may be used to manually calibrate a specified number of captured vehicle interior images. The calibration tool may be a labelImage tool. Specifically, during calibration, the types of targets in the image may be calibrated. For example, if the acquisition device captures a picture of a baby, it is calibrated as a baby, and if the acquisition device captures a picture of a water cup, it is calibrated as a water cup. The above method manually calibrates a specified number of images pre-captured by the acquisition device in order to help measure the accuracy of the calibration of the images output by the training set after the initial model training is completed.

[0044] Furthermore, the calibrated images can be divided into training set images, test set images, and validation set images in a certain ratio. For example, 65% of the calibrated images can be divided into training set images, 25% of the images can be divided into test set images, and the remaining 10% of the images can be divided into validation set images, or the calibrated image data can be divided into training set images, test set images, and validation set images in a ratio of 3:1:1. The specific ratio can be divided according to the model training requirements. Furthermore, the training set images are used to train the model, the test set images are used to test the model, and the validation set images are used to adjust the model, wherein the training set images, the test set images, and the validation set images all include pictures and labels.

[0045] Optionally, the step of inputting the training set images into the initial detection model and training the initial detection model includes: inputting the training set images into the initial detection model in an M-way N-shot manner and training the initial detection model, wherein M and N are positive integers.

[0046] Among them, M-way N-shot is a common experimental setting in meta-learning. M-way means that there are M categories in the training data, and N-shot means that N samples are labeled in each category. For example, 5-way 1-shot can be understood as there are 5 categories in the training data, and 1 sample in these 5 categories is labeled. Optionally, the initial detection model can be a yolov3 model.

[0047] Specifically, the meta-learning training set contains many categories, and each category has multiple samples. The meta-learning training set is the entire data set collected. During the training phase, M categories are randomly selected from the training set, and N samples are selected from each category (a total of M*N data). A meta-learning task is constructed as the training set input of the model; then a batch of samples are extracted from the remaining data in these M categories as the test set of the model. That is, the model is required to learn how to distinguish these M categories from M*N data.

[0048] During the training process, different meta-learning tasks will be sampled for each training, so overall, the training includes a combination of different categories. This mechanism enables the model to learn the common parts of different meta-learning tasks, such as how to extract important features and compare sample similarities. The model trained by this learning mechanism can also better classify the target objects when facing new and unseen meta-learning tasks. Exemplarily, when training the model, only a small number of different types of samples can be used to train the detection model, such as animal samples such as cats and dogs. After the detection model is trained by using a small number of cat and dog samples, when the model is applied to the actual scene, the detection model can recognize the car, tree or other types of objects included in the input image. Through the above method, the use of data is reduced, the application scenarios are increased, and the speed of model training is accelerated.

[0049] Therefore, the calibrated training set images can be input into the initial yolov3 model in a 5-way 1-shot manner, and the initial yolov3 model can be trained to obtain a trained yolov3 model, and then the test set images and the validation set images can be simultaneously input into the trained yolov3 model, and the trained yolov3 model can be trained so that the trained yolov3 model can learn the common features of the training set images, the test set images, and the validation set images, such as the outer contour of the image, the color information of the image, etc. Among them, the types of objects included in the training set images, the test set images, and the validation set images can be different, for example, the training set images can include 5 categories of objects, and the test set images can include 10 objects of different categories from the training set images.

[0050] That is, each time during training, 5 types of target images are selected from the calibrated training set images, and then 1 target image is selected from each type of target image as a small sample training set, which is input into the initial yolov3 model to train the initial yolov3 model. For example, each time five types of images are input, such as images of dogs, cats, birds, babies, and glasses, and then one image is selected from each type as a small sample training set and input into the initial model to train the initial yolov3 model.

[0051] Furthermore, in the process of training the detection model, the size of the input image can be randomly changed, the model can be trained using multi-scale input, and the similarity between the feature vectors of the test set image and the feature vectors of the training set image is calculated one by one to obtain the loss function of the detection model: loss = coordErr + iouErr + clsErr, where coordErr represents the coordinate loss function, iouErr represents the confidence loss function, and claErr represents the classification loss function. The formulas of the three loss functions are as follows:

[0052]

[0053] Furthermore, a weight λcoord=5 is set for coordErr, and λnoobj=0.5 is used to correct iouErr. Then, the difference between the predicted value and the true value can be calculated according to certain rules through the loss function, and then the gradient is obtained through back propagation. The detection model is trained for K iterations based on the loss function, where the K value can be set to any value, such as K>1000, and then the weight of the detection model is updated, and finally the detection model with the highest target recognition rate is obtained.

[0054] Specifically, when training the model, a test image set qc, a training image set sc containing the same c-class objects, and another training image set sn containing different n-class objects are randomly selected to construct a training triplet (qc, sc, sn), where c = n. In the process of training the triplet, only the objects of the c-th class in the training image set are marked as foregrounds, and all other objects are regarded as backgrounds. Therefore, the model not only learns to match objects of the same class between the two image sets (qc, sn), but also can distinguish objects of different classes between the two image sets (qc, sn). However, if there are a large number of background proposals during training, the images usually dominate the samples, especially when there are images with negative support. Therefore, positive and negative sample balance is required when training the model. For foreground proposals and positive training samples (pf, sp), background proposals and positive training samples (pb, sp), and proposals (foreground or background) and negative training samples (pb, sn), the positive and negative sample balance can be balanced in a ratio of 1:2:1, and the matching loss of the selected pairs is calculated by the above loss function.

[0055] Furthermore, the test set images and the validation set images can be input into the trained initial detection model, and the model parameters of the trained initial detection model can be adjusted multiple times by fine-tuning until the error of the trained detection model in identifying the target object is less than a preset value, thereby obtaining a detection model with the highest target object recognition rate, wherein the adjusted model parameters include: dropout ratio, learning speed, and number of layers of the model structure.

[0056] Among them, fine-tuning refers to fine-tuning the parameters of the trained model under a mature framework and a larger training sample, so that the model can adapt to the current scenario. Specifically, the adjustments may include: the dropout ratio, the learning-rate speed, the number of layers of the algorithm structure, etc.

[0057] Specifically, the dropout ratio, learning-rate speed, number of layers of the algorithm structure and other parameters of the trained initial detection model can be adjusted by fine-tuning, so that the trained initial detection model can adapt to the application scenario of the current detection target. When the error of the detection model in identifying the target object is less than the preset error range, the adjustment is stopped to obtain the detection model with the highest target recognition rate.

[0058] The specific adjustment method can be to replace the softmax layer of the initial detection model originally used for single-label multi-classification with a logistic regression layer for multi-label multi-classification, and adjust the model parameters of the trained initial detection model multiple times by fine-tuning. When the loss of the trained detection model in identifying the target object is a reasonably low value, the parameters at this time can be selected as the final parameters of the detection model. Among them, the logistic regression layer mainly uses the sigmoid function, which can constrain the input within the range of 0 to 1. Therefore, when a certain type of output of an image after feature extraction is greater than 0.5 after being constrained by the sigmoid function, it means that it belongs to this class. Furthermore, before training the detection model, the training parameters of the detection model are initialized. Specifically, the maximum number of iterations max_batches can be set to 45,000 times, the image batch size can be set to 64, and the initial learning rate can be set to 0.0001.

[0059] Step 220: Obtain the image of the interior of the vehicle collected by the collection device in real time, and input the image into a pre-built detection model, where the detection model is built based on meta-learning.

[0060] Step 230: Obtain the target object output by the detection model after performing image recognition on the image.

[0061] Step 240: If there is a dangerous object among the objects, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle.

[0062] The detailed explanation of the steps included in step 220, step 230 and step 240 can refer to the corresponding steps in the aforementioned embodiment, which will not be repeated here.

[0063] The present embodiment provides a method for detecting a target object, constructing a detection model, obtaining an image of the interior of a vehicle collected in real time by a collection device, and inputting the image into the detection model, and then identifying the target object outputted after the image is identified according to the acquired detection model. If there is a dangerous target object among the targets, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle. Through the above method, the detection model is trained using small sample data, which reduces the workload of manual calibration. Furthermore, by using fine-tuning to change the parameters of the detection model, the detection model can identify the location and type of the target object at one time, thereby improving the target recognition rate of the detection model.

[0064] See also Figure 4 The present application provides a target object detection method, which is applied to a vehicle, and the method comprises:

[0065] Step 310: Acquire the image collected by the collection device in real time.

[0066] The detailed explanation of the steps included in step 310 can refer to the corresponding steps in the aforementioned embodiment, which will not be repeated here.

[0067] Step 320: Perform image processing on the image to obtain an image with higher definition.

[0068] As a method, the image collected by the acquisition device cannot be used directly because there will be problems such as exposure and noise, so it is necessary to perform image processing on the image collected by the acquisition device. Among them, the image processing of the image can include denoising the image, white balancing the image, and intercepting the image. Then, after performing image processing on the image collected by the acquisition device, an image with less noise, more appropriate white balance ratio, and more targeted can be obtained than the original image. Furthermore, after the original image is white-balanced, the obtained image can have higher clarity than the original image.

[0069] Step 330: Input the image with higher definition into a pre-built detection model.

[0070] Step 340: Obtain the target object output by the detection model after performing image recognition on the image.

[0071] Step 350: If there is a dangerous object among the objects, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle.

[0072] The detailed explanation of the steps included in step 330, step 340 and step 350 can refer to the corresponding steps in the aforementioned embodiment, which will not be repeated here.

[0073] The present embodiment provides a method for detecting a target object, which obtains an image collected by a collection device in real time, processes the image to obtain an image with higher definition, inputs the image with higher definition into a pre-built detection model, obtains the target object output by the detection model after image recognition of the image, and triggers an alarm if there is a dangerous target object among the target objects to prompt the driver to pay attention to the condition inside the vehicle. Through the above method, the detection model can recognize the image with higher definition, which can shorten the time for the detection model to recognize the image, and further, can also improve the recognition rate of the detection model for the target object.

[0074] See also Figure 5 The present application provides a target object detection method, which is applied to a vehicle, and the method comprises:

[0075] Step 410: Obtain the image of the interior of the vehicle collected by the collection device in real time, and input the image into a pre-built detection model, where the detection model is built based on meta-learning.

[0076] Step 420: Obtain the target object output by the detection model after performing image recognition on the image.

[0077] The detailed explanation of the steps included in step 410 and step 420 can refer to the corresponding steps in the aforementioned embodiment, which will not be repeated here.

[0078] Step 430: If the target objects include a first target object, obtain position information of a reference target object, where the reference target object is a target object other than the first target object included in the vehicle.

[0079] As a method, the first target object can be understood as the target object that needs to be detected first in the current application scenario. For example, in an application scenario where a baby sitting alone in the back seat of a car may encounter danger, the baby can be used as the first target object.

[0080] When it is detected that the first target object is included in the target objects included in the vehicle, the target objects within the specified range of the first target object are detected.

[0081] Among them, the target object within the specified range of the first target object can be determined according to the position information relationship between different targets.

[0082] Specifically, the position information of the reference target object in the image and the position information of the first target object in the image output by the detection model can be obtained, and then the target object within the specified range of the first target object can be determined based on the relationship between the position information of the reference target object and the position information of the first target object.

[0083] Exemplarily, after determining that the first target object is a baby, the baby's location information is obtained. Further, the location information of objects other than the baby can be obtained, and then the objects within the specified range of the baby are determined based on the relationship between the location information of the baby and other objects.

[0084] Step 440: taking a reference target whose corresponding position information and the position information of the first target have a difference within a preset range as a second target, performing image recognition on the second target, and detecting whether there is a dangerous target among the second target.

[0085] As a method, the preset range can be understood as a preset difference range of position information between different targets. Further, the position information of the obtained multiple reference targets is respectively subjected to difference calculation with the position information of the first target to obtain multiple differences, and it is determined whether the multiple differences meet the preset difference range, and the reference target whose corresponding position information and the position information of the first target have a difference within the preset range is used as the second target.

[0086] After the second target object is determined by the above method, the vehicle performs image recognition on the second target object to detect whether there is a dangerous target object in the second target object.

[0087] Step 450: If there is a dangerous object among the second objects, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle.

[0088] As one method, the step of triggering an alarm if there is a dangerous object among the second objects to prompt the driver to pay attention to the situation inside the vehicle includes: detecting whether there is a danger sign among the objects included in the second objects; if there is the danger sign, treating the object in the second objects corresponding to the danger sign as a dangerous object, and triggering an alarm to prompt the driver to pay attention to the situation inside the vehicle.

[0089] The danger sign may be a danger pattern sign on the second target object, such as a water cup with a "Caution: Hot Hands" pattern printed on it. Further, the vehicle may detect whether there is a target with a danger sign among the second targets. If a target with a danger sign is detected, it is determined that the target may cause the first target object to be in danger, and therefore, an alarm may be triggered to prompt the driver to pay attention to the situation inside the vehicle.

[0090] For example, if it is detected that there is only a child in the back seat of the vehicle without an adult accompanying him, the system can focus on detecting and identifying the objects around the child. If a dangerous sign is detected on a cup or other object around the child, the system can trigger a reminder for the driver to pay attention to the child in the back seat of the vehicle. This can reduce the risk of the driver being distracted by the need to pay attention to the child in the back seat at all times, and also protect the safety of the child.

[0091] The present embodiment provides a method for detecting an object, which obtains an image of the interior of the vehicle collected in real time by a collection device, and inputs the image into a pre-built detection model, wherein the detection model is built based on meta-learning, and obtains the object output by the detection model after image recognition of the image. If the object includes a first object, the position information of the reference object is obtained, wherein the reference object is an object other than the first object included in the interior of the vehicle, and the reference object whose corresponding position information and the position information of the first object have a difference within a preset range is used as the second object, and image recognition is performed on the second object to detect whether there is a dangerous object in the second object. If there is a dangerous object in the second object, an alarm is triggered to prompt the driver to pay attention to the condition inside the vehicle. Through the above method, the driver can pay attention to the condition inside the vehicle in a timely manner, thereby reducing the occurrence of traffic accidents caused by the driver paying too much attention to the condition inside the vehicle.

[0092] See also Figure 6 The present application provides a target detection device 500, which includes an image input unit 510, a target acquisition unit 520 and an alarm unit 530.

[0093] The image input unit 510 is used to obtain the image of the interior of the vehicle collected by the collection device in real time, and input the image into a pre-built detection model, where the detection model is built based on meta-learning.

[0094] Specifically, the image input unit is used to obtain the image collected by the collection device in real time; perform image processing on the image to obtain an image with higher definition; and input the image with higher definition into a pre-constructed detection model.

[0095] The target object acquisition unit 520 is used to acquire the target object output after the detection model performs image recognition on the image.

[0096] The alarm unit 530 is used to trigger an alarm if there is a dangerous object among the objects, so as to prompt the driver to pay attention to the situation inside the vehicle.

[0097] The alarm unit 530 is used to obtain the position information of the reference target if the target includes the first target, and the reference target is the target included in the vehicle except the first target; the reference target whose corresponding position information and the position information of the first target have a difference within a preset range is used as the second target, and the image recognition is performed on the second target to detect whether there is a dangerous target among the second targets. If there is a dangerous target among the second targets, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle.

[0098] Optionally, the alarm unit 530 is also used to detect whether there is a danger sign among the targets included in the second target object; if there is a danger sign, the target object corresponding to the danger sign in the second target object is regarded as a dangerous target, and an alarm is triggered to prompt the driver to pay attention to the condition inside the vehicle.

[0099] like Figure 7 As shown, the device 500 also includes:

[0100] The model building unit 540 is used to obtain a specified number of images of the interior of the vehicle that have been previously collected by the collection device; calibrate the images to obtain calibrated images; divide the calibrated images into training set images, test set images, and validation set images; input the training set images into the initial detection model to train the initial detection model; input the test set images and the validation set images into the trained initial detection model to obtain a loss function for the target object recognition rate, and adjust the model parameters of the trained initial detection model multiple times based on the loss function to obtain a detection model with the highest target object recognition rate, and use the detection model with the highest target object recognition rate as the pre-built detection model.

[0101] The model building unit 540 is further used to input the training set images into the initial detection model in an M-way N-shot manner to train the initial detection model, wherein M and N are positive integers.

[0102] Furthermore, the model building unit 540 is also used to input the test set images and the validation set images into the trained initial detection model, and adjust the model parameters of the trained initial detection model multiple times by fine-tuning until the error of the trained detection model in identifying the target object is less than a preset value, thereby obtaining a detection model with the highest target object recognition rate, wherein the adjusted model parameters include: dropout ratio, learning speed, and number of layers of the model structure.

[0103] It should be noted that those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In several embodiments provided in the present application, the coupling between modules can be electrical, mechanical or other forms of coupling. In addition, each functional module in each embodiment of the present application can be integrated in a processing module, or each module can exist physically separately, or two or more modules can be integrated in one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules.

[0104] The following will be combined Figure 8 A vehicle provided in the present application is described.

[0105] See also Figure 8 Based on the above-mentioned object detection method and device, the embodiment of the present application also provides another vehicle 100 that can execute the above-mentioned object detection method. The vehicle 100 in the present application may include one or more (only one is shown in the figure) processors 102, a memory 104 and a wireless module 106 coupled to each other. Among them, the memory 104 stores a program that can execute the content of the above-mentioned embodiment, and the processor 102 can execute the program stored in the memory 104.

[0106] Among them, the processor 102 may include one or more processing cores. The processor 102 uses various interfaces and lines to connect various parts of the entire vehicle 100, and executes various functions and processes data of the vehicle 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 104, and calling data stored in the memory 104. Optionally, the processor 102 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 102 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 102, but may be implemented separately through a communication chip.

[0107] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the vehicle 100 during use (such as a phone book, audio and video data, chat record data), etc.

[0108] The wireless module 106 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electric signals, so as to communicate with a communication network or other equipment, for example, communicate with an audio playback device. The wireless module 106 may include various existing circuit elements for performing these functions, for example, an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a user identity module (SIM) card, a memory, etc. The wireless module 106 may communicate with various networks such as the Internet, an intranet, a wireless network, or communicate with other equipment via a wireless network. The above-mentioned wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network.

[0109] Please refer to Fig. 9 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0110] The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in an appropriate form.

[0111] In summary, the present application provides a method, device, vehicle and storage medium for detecting a target object. First, an image of the interior of the vehicle collected in real time by a collection device is obtained, and the image is input into a pre-built detection model, which is built based on meta-learning. Then, the target object output by the detection model after image recognition is obtained. If there is a dangerous target among the targets, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle. Through the above method, the pre-built meta-learning-based detection model can be used to identify targets in the vehicle with high precision, thereby improving the recognition rate of the targets. In addition, when a dangerous target is detected, an alarm is triggered to prompt the driver to pay attention to the conditions inside the vehicle, which can reduce the occurrence of traffic accidents.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting a target object, characterized in that: Applied to a vehicle, the method comprises: Acquire an image of the interior of the vehicle acquired in real time by an acquisition device, and input the image into a pre-built detection model, wherein the detection model is built based on meta-learning; Obtaining a target object outputted by the detection model after performing image recognition on the image; If there is a dangerous object among the objects, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle, wherein if there is a dangerous object among the objects, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle, including: If the target objects include a first target object, obtaining position information of a reference target object, wherein the reference target object is a target object other than the first target object included in the vehicle; Taking a reference target whose corresponding position information and the position information of the first target have a difference within a preset range as a second target, performing image recognition on the second target, and detecting whether there is a dangerous target among the second target; If there is a dangerous object among the second objects, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle.

2. The method according to claim 1, characterized in that Before inputting the image of the interior of the vehicle collected in real time by the collection device into the pre-built detection model, the method further includes: Acquire a specified number of vehicle interior images previously acquired by the acquisition device; Calibrate the image, classify images with the same semantics in the image into one category, and obtain a calibrated image; Dividing the calibrated images into training set images, test set images, and validation set images in proportion, wherein the training set images, the test set images, and the validation set images include images of different types; Inputting the training set images into an initial detection model to train the initial detection model; The test set images and the validation set images are input into the trained initial detection model to obtain a loss function of the target object recognition rate. Based on the loss function, the model parameters of the trained initial detection model are adjusted multiple times to obtain a detection model with the highest target object recognition rate. The detection model with the highest target object recognition rate is used as the pre-built detection model, wherein the expression of the loss function is loss=coordErr+iouErr+clsErr, wherein loss represents the loss function, coordErr represents the coordinate loss function, iouErr represents the confidence loss function, and claErr represents the category loss function.

3. The method according to claim 2, characterized in that The step of inputting the training set images into an initial detection model to train the initial detection model includes: The training set images are input into the initial detection model in an M-way N-shot manner to train the initial detection model, wherein M and N are positive integers.

4. The method according to claim 2, characterized in that: The test set images and the validation set images are input into the trained initial detection model to obtain a loss function of the target object recognition rate, and the model parameters of the trained initial detection model are adjusted multiple times based on the loss function to obtain a detection model with the highest target object recognition rate, and the detection model with the highest target object recognition rate is used as the pre-established detection model, including: The test set images and the validation set images are input into the trained initial detection model, and the model parameters of the trained initial detection model are adjusted multiple times by fine-tuning until the error of the trained detection model in identifying the target object is less than a preset value, thereby obtaining a detection model with the highest target object recognition rate, wherein the adjusted model parameters include: dropout ratio, learning speed, and number of layers of the model structure.

5. The method according to claim 1, characterized in that The acquiring device acquires the image of the interior of the vehicle in real time, and inputs the image into a pre-built detection model, including: Acquire the image collected by the collection device in real time; Performing image processing on the image to obtain an image with higher definition; The image with higher definition is input into a pre-built detection model.

6. The method according to claim 1, characterized in that If there is a dangerous object among the second objects, triggering an alarm to prompt the driver to pay attention to the situation inside the vehicle includes: Detecting whether there is a danger sign among the objects included in the second object; If there is the danger sign, the object corresponding to the danger sign in the second object is regarded as a danger object, and an alarm is triggered to prompt the driver to pay attention to the condition inside the vehicle.

7. A target object detection device, characterized in that: Applied to a vehicle, the device comprises: An image input unit, used to obtain the image of the interior of the vehicle collected by the collection device in real time, and input the image into a pre-built detection model, wherein the detection model is built based on meta-learning; A target object acquisition unit, used for acquiring a target object outputted after the detection model performs image recognition on the image; An alarm unit is used to trigger an alarm if there is a dangerous object among the objects, so as to prompt the driver to pay attention to the situation inside the vehicle, wherein the alarm unit includes: If the target objects include a first target object, obtaining position information of a reference target object, wherein the reference target object is a target object other than the first target object included in the vehicle; Taking a reference target whose corresponding position information and the position information of the first target have a difference within a preset range as a second target, performing image recognition on the second target, and detecting whether there is a dangerous target among the second target; If there is a dangerous object among the second objects, an alarm is triggered to prompt the driver to pay attention to the situation inside the vehicle.

8. A vehicle, characterized in that: including one or more processors and memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the method according to any one of claims 1 to 6.

Images