Target state recognition method, device, equipment, storage medium and vehicle

Abstract

The present disclosure relates to a target state recognition method, device, equipment, storage medium and vehicle. In the embodiment of the present disclosure, a road image in a driving direction of a target vehicle is acquired; a target tracking frame of a target in the road image is acquired based on the road image, and target pixels corresponding to the target contained in the target tracking frame and other pixels corresponding to other targets except the target contained in the target tracking frame are acquired, the target including a pedestrian and a cyclist; the target pixels contained in the target tracking frame are compared with the other pixels contained in the target tracking frame to determine a state of the target, a plurality of different states of the target can be obtained, the types of the target state are increased, the accuracy of the target state recognition is improved, more target state information can be provided for intelligent driving, and the safety of the intelligent driving is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image recognition, and particularly relates to a target state recognition method and device, equipment, a storage medium and a vehicle. BACKGROUND

[0002] It is crucial to recognize the state of a pedestrian or a cyclist in a road in intelligent driving, and the vehicle can determine the next driving action according to the state of the pedestrian or the cyclist, so as to ensure the driving safety of the vehicle.

[0003] At present, the state of the pedestrian or the cyclist is mainly recognized according to an action recognition model, but the state of the pedestrian or the cyclist recognized by the action recognition model mainly includes a walking speed and a walking direction, and the type of the recognized state is relatively single, and the accuracy is low, which cannot meet the current demand of intelligent driving. SUMMARY

[0004] In order to solve the above technical problems, the present disclosure provides a target state recognition method, device, equipment, a storage medium and a vehicle.

[0005] A first aspect of the embodiments of the present disclosure provides a target state recognition method, which comprises:

[0006] obtaining a road image in a driving direction of a target vehicle;

[0007] based on the road image, obtaining a target tracking frame of a target in the road image, and target pixels corresponding to the target contained in the target tracking frame and other pixels corresponding to other targets other than the target contained in the target tracking frame, the target including a pedestrian and a cyclist;

[0008] comparing the target pixels contained in the target tracking frame with the other pixels contained in the target tracking frame to determine the state of the target.

[0009] A second aspect of the embodiments of the present disclosure provides a target state recognition device, which comprises:

[0010] a first obtaining module, configured to obtain a road image in a driving direction of a target vehicle;

[0011] a second obtaining module, configured to obtain, based on the road image, a target tracking frame of a target in the road image, and target pixels corresponding to the target contained in the target tracking frame and other pixels corresponding to other targets other than the target contained in the target tracking frame, the target including a pedestrian and a cyclist;

[0012] a first determining module, configured to compare the target pixels contained in the target tracking frame with the other pixels contained in the target tracking frame to determine the state of the target.

[0013] A third aspect of the embodiments of the present disclosure provides a vehicle terminal, the vehicle terminal comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the target state recognition method of the first aspect can be implemented.

[0014] A fourth aspect of the embodiments of the present disclosure provides a computer readable storage medium, the storage medium stores a computer program, and when the computer program is executed by the processor, the target state recognition method of the first aspect can be implemented.

[0015] A fifth aspect of the embodiments of the present disclosure provides a vehicle, the vehicle comprising the vehicle terminal of the third aspect.

[0016] The technical solutions provided by the embodiments of the present disclosure have the following advantages compared with the prior art:

[0017] In the embodiments of the present disclosure, the road image in the driving direction of the target vehicle is acquired; based on the road image, the target tracking frame of the target in the road image is acquired, and the target pixel corresponding to the target contained in the target tracking frame and the other pixel corresponding to other targets other than the target contained in the target tracking frame are acquired, the target includes pedestrians and cyclists; the target pixel contained in the target tracking frame is compared with the other pixel contained in the target tracking frame to determine the state of the target, a plurality of different states of the target can be obtained, the types of target states are increased, the accuracy of target state recognition is improved, more target state information can be provided for intelligent driving, and the safety of intelligent driving is improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure together with the specification.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0020] Figure 1 is a flowchart of a target state recognition method provided by the embodiments of the present disclosure;

[0021] Figure 2 is a flowchart of another target state recognition method provided by the embodiments of the present disclosure;

[0022] Figure 3 is a flowchart of another target state recognition method provided by the embodiments of the present disclosure;

[0023] Figure 4 is a flowchart of another target state recognition method provided by an embodiment of the present disclosure;

[0024] Figure 5 is a flowchart of another target state recognition method provided by an embodiment of the present disclosure;

[0025] Figure 6 is a flowchart of another target state recognition method provided by an embodiment of the present disclosure;

[0026] Figure 7 is a structural schematic diagram of a target state recognition device provided by an embodiment of the present disclosure;

[0027] Figure 8 is a structural schematic diagram of a vehicle-mounted terminal provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the schemes of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, a lot of specific details are set forth in order to give a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other different manners from those described herein; obviously, the embodiments described in the specification are only some embodiments of the present disclosure, not all the embodiments.

[0030] It should be understood that each step recorded in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] It should be noted that, in this document, relationship terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Moreover, the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0032] It should be noted that the modification of "one" and "multiple" mentioned in the present disclosure is illustrative but not restrictive, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0033] Identifying the state of a pedestrian or a cyclist in a road is crucial in intelligent driving, and the vehicle can determine the next driving action according to the state of the pedestrian or the cyclist, thereby ensuring the safety of vehicle driving.

[0034] At present, the state of a pedestrian or a cyclist is mainly identified according to an action recognition model, but the state of a pedestrian or a cyclist identified by the action recognition model mainly includes walking speed, walking direction, etc., the type of the identified state is relatively single, and the accuracy is low, which cannot meet the current demand of intelligent driving.

[0035] In view of the defects of the related art in pedestrian and cyclist state identification, the embodiments of the present disclosure provide a target state identification method, device, equipment, storage medium and vehicle, which can determine a plurality of different states of a target according to target pixels of the target contained in a target tracking frame and other pixels of other targets other than the target, increase the type of target state, improve the accuracy of target state identification, provide more target state information for intelligent driving, and improve the safety of intelligent driving.

[0036] The target state identification method provided by the embodiments of the present disclosure can be executed by a vehicle terminal, which can be understood as any device with processing and computing capabilities, and can be used to control various parameters of a vehicle to enable intelligent driving of the vehicle.

[0037] In order to better understand the inventive concept of the embodiments of the present disclosure, the technical solutions of the embodiments of the present disclosure will be described below in conjunction with exemplary embodiments.

[0038] Figure 1 is a flowchart of a target state identification method provided by the embodiments of the present disclosure, as Figure 1 shown, the target state identification method provided by the embodiments of the present disclosure can include steps 110-130:

[0039] Step 110, acquiring a road image in the driving direction of a target vehicle.

[0040] In the embodiments of the present disclosure, a vehicle camera is installed on the target vehicle, which can collect a road image in the driving direction of the target vehicle, and a vehicle terminal installed on the target vehicle can acquire the road image in the driving direction of the target vehicle from the vehicle camera.

[0041] In step 120, a target tracking box of a target in the road image is obtained based on the road image, and target pixels corresponding to the target in the target tracking box and other pixels corresponding to other targets outside the target in the target tracking box are obtained, the target including a pedestrian and a cyclist.

[0042] In the embodiments of the present disclosure, the target includes a pedestrian and a cyclist, the cyclist can be understood as a person pushing or riding a non-motor vehicle, the target tracking box can include a pedestrian tracking box and a cyclist tracking box, the pedestrian tracking box can be understood as a minimum bounding box of a pedestrian in the image, and the cyclist tracking box can be understood as a minimum bounding box of a cyclist in the image.

[0043] In the embodiments of the present disclosure, based on the road image, the target tracking box of the target in the road image is obtained, which can be obtained based on a preset convolutional neural model, the preset convolutional neural model can be understood as a convolutional neural network, the convolutional neural model includes a target detection model, the target detection model can identify pedestrians and cyclists in the road image, and assign a pedestrian tracking box to each pedestrian and a cyclist tracking box to each cyclist.

[0044] In the embodiments of the present disclosure, the pixel can be understood as a pixel point in the road image, the target pixel can be understood as a pixel point constituting a target image, and the other pixel can be understood as a pixel point constituting other targets.

[0045] In the embodiments of the present disclosure, based on the road image, the target tracking box of the target in the road image is obtained, which can be obtained based on a preset convolutional neural model, the preset convolutional neural model can be understood as a convolutional neural network, the convolutional neural model includes a target detection model, the target detection model can identify pedestrians and cyclists in the road image, and assign a pedestrian tracking box to each pedestrian and a cyclist tracking box to each cyclist.

[0046] In the embodiments of the present disclosure, the other target can include other pedestrians, other cyclists, target vehicles, other vehicles, road driving areas, road non-driving areas, lanes, pedestrian crossings, fences, buildings, traffic lights, traffic signs, vegetation, parking poles, ground locks, columns, etc., but is not limited thereto. Among them, the road area in the road image can be divided into a road driving area and a road non-driving area, the road driving area can be understood as a road area for motor vehicle driving, and the road non-driving area can be understood as a road area for non-motor vehicle or pedestrian driving.

[0047] In the embodiment of the present disclosure, after the vehicle terminal obtains the road image, the vehicle terminal can obtain a target tracking box of a target in the road image based on the road image, and obtain target pixels corresponding to the target in the target tracking box and other pixels corresponding to other targets in the target tracking box.

[0048] In the embodiment of the present disclosure, the target tracking box can include at least one of a pedestrian tracking box, a cyclist tracking box, and a cyclist fusion box, the cyclist fusion box including the pedestrian tracking box and the cyclist tracking box, and the cyclist fusion box being fused from the pedestrian tracking box and the cyclist tracking box.

[0049] In step 130, the target pixels in the target tracking box are compared with the other pixels in the target tracking box to determine the state of the target.

[0050] In the embodiment of the present disclosure, the state of the target can include that the target is blocked by other targets, the target is in a road driving area, and the like, but is not limited thereto. The target being blocked by other targets can be understood as the target being located behind the other targets.

[0051] In the embodiment of the present disclosure, after the vehicle terminal obtains the target tracking box and the target pixels and the other pixels in the target tracking box, the vehicle terminal can compare the target pixels in the target tracking box with the other pixels in the target tracking box to determine the state of the target.

[0052] In the embodiment of the present disclosure, by obtaining a road image in a driving direction of a target vehicle, obtaining a target tracking box of a target in the road image based on the road image, and obtaining target pixels corresponding to the target in the target tracking box and other pixels corresponding to other targets in the target tracking box, the target including a pedestrian and a cyclist, comparing the target pixels in the target tracking box with the other pixels in the target tracking box to determine the state of the target, a plurality of different states of the target can be obtained, the types of the states of the target are increased, the accuracy of the state recognition of the target is improved, more state information of the target can be provided for intelligent driving, and the safety of the intelligent driving is improved.

[0053] Figure 2 is a flowchart of a target state recognition method provided by the embodiment of the present disclosure, as shown in Figure 2 The target state recognition method provided by the embodiment can include steps 210-250.

[0054] In step 210, a road image in a driving direction of a target vehicle is obtained.

[0055] In step 220, a target tracking box of the target in the road image is obtained based on the road image, and target pixels corresponding to the target contained in the target tracking box and other pixels corresponding to other targets except the target contained in the target tracking box are obtained, the target including a pedestrian and a cyclist.

[0056] Steps 210-220 in the embodiments of the present disclosure can refer to the content of steps 110-120 described above, which will not be repeated here.

[0057] In step 230, the number of target pixels contained in the target tracking box and the number of other pixels are counted.

[0058] In the embodiments of the present disclosure, after obtaining the target tracking box of the target in the road image and the target pixels and other pixels contained in the target tracking box, the vehicle terminal can count the number of target pixels and the number of other pixels contained in the target tracking box.

[0059] In step 240, a first ratio between the number of other pixels and the number of target pixels is calculated.

[0060] In the embodiments of the present disclosure, after obtaining the number of target pixels and the number of other pixels in the target tracking box, the vehicle terminal can calculate the first ratio between the number of other pixels and the number of target pixels.

[0061] In step 250, if the first ratio is greater than a first preset threshold, it is determined that the state of the target is that the target is occluded by the other target.

[0062] In the embodiments of the present disclosure, if the first ratio between the number of other pixels and the number of target pixels in the target tracking box is greater than the first preset threshold, the vehicle terminal can determine that the position of the target is behind the other target, and can determine that the state of the target is that the target is occluded by the other target, if the first ratio is less than or equal to the first preset threshold, the vehicle terminal can determine that the state of the target is that the target is not occluded by the other target, wherein the first preset threshold can be obtained by actual test, for example, 0.5, which is not limited here.

[0063] For example, if the target in the target tracking box is a pedestrian, and the other target contained in the target tracking box is a fence, if the first ratio between the number of fence pixels and the number of pedestrian pixels in the target tracking box is greater than the first preset threshold, the vehicle terminal can determine that the pedestrian is occluded by the fence; for another example, if the target in the target tracking box is a cyclist, and the other target contained in the target tracking box is another vehicle, if the first ratio between the number of other vehicle pixels and the number of cyclist pixels in the target tracking box is greater than the first preset threshold, the vehicle terminal can determine that the cyclist is occluded by the other vehicle.

[0064] Therefore, whether the target is blocked by other targets can be determined by the number of target pixels and the number of other pixels contained in the target tracking frame, the types of target states are increased, and the accuracy of target state recognition is improved.

[0065] Figure 3 is a flowchart of a target state recognition method provided by an embodiment of the present disclosure, as shown in the figure, the target state recognition method provided by the embodiment can include steps 310-350. Figure 3

[0066] Step 310, obtain a road image in a driving direction of a target vehicle.

[0067] Step 320, based on the road image, obtain a target tracking frame of a target in the road image, and target pixels corresponding to the target contained in the target tracking frame and other pixels corresponding to other targets except the target contained in the target tracking frame, the target including a pedestrian and a cyclist.

[0068] Steps 310-320 in the embodiment of the present disclosure can refer to the content of steps 110-120 described above, which will not be repeated here.

[0069] Step 330, count the number of target pixels contained in the target tracking frame and the number of pixels of a road driving area.

[0070] In the embodiment of the present disclosure, the semantic segmentation model included in the preset convolutional neural model in the vehicle terminal can divide the road area in the road image into a road driving area and a road non-driving area, the road driving area can be understood as a road area for motor vehicle driving, and the road non-driving area can be understood as a road area for non-motor vehicle or pedestrian driving.

[0071] In the embodiment of the present disclosure, after the vehicle terminal obtains the target tracking frame of the target in the road image and the target pixels and other pixels contained in the target tracking frame, the vehicle terminal can count the number of target pixels contained in the target tracking frame and the number of pixels of the road driving area.

[0072] Step 340, calculate a second ratio between the number of pixels of the road driving area and the number of target pixels.

[0073] In the embodiment of the present disclosure, after the vehicle terminal obtains the number of target pixels contained in the target tracking frame and the number of pixels of the road driving area, the vehicle terminal can calculate a second ratio between the number of pixels of the road driving area and the number of target pixels.

[0074] Step 350, if the second ratio is greater than a second preset threshold, determine that the state of the target is that the target is in the road driving area.

[0075] ​In the embodiments of the present disclosure, if the second ratio between the number of pixels of the road driving area and the number of target pixels is greater than a second preset threshold, the vehicle terminal can determine that the state of the target is that the target is in the road driving area. The second preset threshold can be obtained through actual test, which is not limited here.

[0076] Therefore, by the number of target pixels contained in the target tracking frame and the number of pixels of the road driving area, it can be determined whether the target is in the road driving area, the types of target states are increased, and the accuracy of target state recognition is improved.

[0077] Figure 4 is a flowchart of a target state recognition method provided by the embodiments of the present disclosure, as shown in Figure 4 The target state recognition method provided by the embodiments of the present disclosure can include steps 410-430.

[0078] Step 410, obtaining a road image of a driving direction of a target vehicle.

[0079] Step 420, based on the road image, obtaining a target tracking frame of a target in the road image, the target including a pedestrian and a cyclist.

[0080] Steps 410-420 in the embodiments of the present disclosure can refer to the content of steps 110-120 described above, which will not be repeated here.

[0081] Step 430, determining the state of the target based on the position of the target tracking frame in the road image.

[0082] In the embodiments of the present disclosure, after the vehicle terminal obtains the target tracking frame of the target in the road image, the state of the target can be determined based on the position of the target tracking frame in the road image.

[0083] In some embodiments, determining the state of the target based on the position of the target tracking frame in the road image can include steps 4301-4302.

[0084] Step 4301, determining the position of the midpoint pixel of the lower frame line of the target tracking frame in the road image.

[0085] In the embodiments of the present disclosure, the target tracking frame is a rectangular frame, and according to the standing posture of the pedestrian, the frame line corresponding to the head of the pedestrian is the upper frame line, the frame line corresponding to the feet of the pedestrian is the lower frame line, the frame line corresponding to the left arm of the pedestrian is the left frame line, and the frame line corresponding to the right arm of the pedestrian is the right frame line.

[0086] In the embodiments of the present disclosure, the vehicle terminal can determine the position of the midpoint pixel of the lower frame line of the target tracking frame in the road image, that is, the pixel coordinates of the midpoint pixel of the lower frame line in the image coordinate system.

[0087] Step 4302, determining the state of the target based on the position of the middle point pixel of the lower frame line of the target tracking frame in the road image.

[0088] In the embodiments of the present disclosure, the vehicle terminal can determine the state of the target based on the position of the middle point pixel of the lower frame line of the target tracking frame in the road image.

[0089] In some embodiments, determining the state of the target based on the position of the middle point pixel of the lower frame line of the target tracking frame in the road image can include steps 430201-430203:

[0090] Step 430201, if the position of the middle point pixel of the lower frame line is in a first preset pixel area at the bottom of the road image, determining that the state of the target is that the target is in the close distance of the target vehicle.

[0091] In the embodiments of the present disclosure, the first preset pixel area can be understood as an area determined by a plurality of rows of pixels at the bottom of the road image, and the first preset pixel area can be set as needed, which is not specifically limited here.

[0092] In the embodiments of the present disclosure, if the position of the middle point pixel of the lower frame line of the target tracking frame is in the first preset pixel area at the bottom of the road image, the vehicle terminal can determine that the target is close to the target vehicle, and can determine that the state of the target is that the target is in the close distance of the target vehicle.

[0093] Step 430202, if the distance between the position of the middle point pixel of the lower frame line and the vanishing point pixel in the road image is less than a preset distance threshold, determining that the state of the target is that the target is in the far distance of the target vehicle.

[0094] The vanishing point pixel in the embodiments of the present disclosure can be understood as a pixel point corresponding to the far end of the target vehicle in the driving direction of the target vehicle converging to a point in the road image. It can be understood that the distance between the vanishing point pixel and the target vehicle collecting the road image is far.

[0095] In the embodiments of the present disclosure, if the distance between the position of the middle point pixel of the lower frame line of the target tracking frame and the vanishing point pixel in the road image is less than a preset distance threshold, the vehicle terminal can determine that the target is far from the target vehicle, and can determine that the state of the target is that the target is in the far distance of the target vehicle.

[0096] Step 430203, if the position of the middle point pixel of the lower frame line is in a second preset pixel area at the lower left corner or the lower right corner of the road image, determining that the state of the target is that the target cuts into the picture.

[0097] In the embodiments of the present disclosure, the second preset pixel region can be understood as a region determined by multiple rows of pixels and multiple columns of pixels at the lower left corner or the lower right corner of the road image, and the second preset pixel region can be set as needed, for example, 20 rows x 10 columns, which is not limited here.

[0098] In the embodiments of the present disclosure, if the position of the lower frame line midpoint pixel of the target tracking frame is in the second preset pixel region at the lower left corner or the lower right corner of the road image, the vehicle terminal can determine that the state of the target is that the target cuts into the picture. The target cutting into the picture can indicate that the target is very close to the target vehicle, and the target vehicle needs to slow down or avoid in advance to ensure the safety of the target.

[0099] Therefore, the state of the target can be determined by the position of the lower frame line midpoint pixel of the target tracking frame in the road image, including whether the target is in the close distance of the target vehicle, whether the target is in the far distance of the target vehicle, and whether the target cuts into the picture, thereby increasing the types of target states and improving the accuracy of target state recognition.

[0100] Figure 5 FIG. 1 is a flowchart of a target state recognition method provided by the embodiments of the present disclosure, as shown in FIG. 1, the target state recognition method provided by the embodiments of the present disclosure can include steps 510-570. Figure 5

[0101] Step 510, acquire a road image in the driving direction of a target vehicle.

[0102] Step 520, based on the road image, acquire a target tracking frame of a target in the road image, the target tracking frame including at least one of a pedestrian tracking frame, a cyclist tracking frame, and a cyclist fusion frame.

[0103] Steps 510-520 in the embodiments of the present disclosure can refer to the content of steps 110-120 described above, which will not be repeated here.

[0104] Step 530, when both the pedestrian tracking frame and the cyclist tracking frame exist in the target tracking frame, acquire a longitudinal distance between the pedestrian tracking frame and the cyclist tracking frame.

[0105] The longitudinal distance in the embodiments of the present disclosure can be understood as a direction parallel to the driving direction of the target vehicle.

[0106] In the embodiments of the present disclosure, when both the pedestrian tracking frame and the cyclist tracking frame exist in the target tracking frame, the vehicle terminal can acquire the longitudinal distance between the pedestrian tracking frame and the cyclist tracking frame from the storage space.

[0107] Step 540, if the longitudinal distance is greater than a preset distance threshold, determine the cyclist tracking frame as an adjusted target tracking frame.

[0108] ​In the embodiment of the present disclosure, if the longitudinal distance is greater than the preset distance threshold, the vehicle terminal can determine the cyclist tracking frame as the adjusted target tracking frame. The preset distance threshold can be set according to actual needs, which is not limited here.

[0109] Step 550, if the longitudinal distance is less than or equal to the preset distance threshold, the pedestrian tracking frame and the cyclist tracking frame are fused to generate a cyclist fusion frame.

[0110] In the embodiment of the present disclosure, if the longitudinal distance is less than or equal to the preset distance threshold, the vehicle terminal can fuse the pedestrian tracking frame and the cyclist tracking frame to generate a cyclist fusion frame, which can include the pedestrian tracking frame and the cyclist tracking frame at the same time.

[0111] Step 560, the cyclist fusion frame is determined as the adjusted target tracking frame.

[0112] In the embodiment of the present disclosure, after generating the cyclist fusion frame, the vehicle terminal can determine the cyclist fusion frame as the adjusted target tracking frame.

[0113] Step 570, comparing the target pixel included in the adjusted target tracking frame with other pixels included in the adjusted target tracking frame to determine the state of the target.

[0114] In the embodiment of the present disclosure, after obtaining the adjusted target tracking frame, the vehicle terminal can compare the target pixel included in the adjusted target tracking frame with other pixels included in the adjusted target tracking frame to determine the state of the target. The specific steps can refer to the related content in the above step 130, which will not be repeated here.

[0115] Therefore, by adjusting the target tracking frame, the matching degree between the target tracking frame and the actual target can be improved, and the accuracy of target state recognition can be improved.

[0116] Figure 6 is a flowchart of a target state recognition method provided by the embodiment of the present disclosure, as shown in Figure 6 The target state recognition method provided by the embodiment can include steps 610-680.

[0117] Step 610, obtaining a road image in the driving direction of a target vehicle.

[0118] Step 620, based on the road image, obtaining a target tracking frame of a target in the road image, and target pixels corresponding to the target in the target tracking frame and other pixels corresponding to other targets except the target in the target tracking frame, the target including pedestrians and cyclists.

[0119] Step 630, comparing the target pixel contained in the target tracking frame with other pixels contained in the target tracking frame to determine the state of the target.

[0120] The steps 610-630 in the embodiments of the present disclosure can refer to the content of the steps 110-130 described above, which will not be repeated here.

[0121] Step 640, determining the state of the target as the first state.

[0122] In the embodiments of the present disclosure, after obtaining the state of the target in the road image, the vehicle-mounted terminal can store the road image and the state of the target in the road image, and determine the state of the target as the first state.

[0123] Step 650, for each first state, obtaining a preset number of frames of road images corresponding to the first state.

[0124] In the embodiments of the present disclosure, for each first state, the vehicle-mounted terminal can obtain a preset number of frames of road images corresponding to the first state from the storage space. The preset number can be set according to actual needs, which will not be specifically limited here.

[0125] Step 660, comparing the target pixel contained in the target tracking frame in each road image with other pixels contained in the target tracking frame to determine the second state of the target in each road image.

[0126] In the embodiments of the present disclosure, after obtaining the preset number of frames of road images corresponding to the first state, the vehicle-mounted terminal can compare the target pixel contained in the target tracking frame in each road image with other pixels contained in the target tracking frame to determine the second state of the target in each road image. The specific content can refer to the related content in the step 130 described above, which will not be repeated here.

[0127] Step 670, counting the number of states in the second state that are the same as the first state.

[0128] In the embodiments of the present disclosure, after obtaining the second state of the target in each road image, the vehicle-mounted terminal can count the number of states in the second state that are the same as the first state.

[0129] Step 680, if a third proportion of the number of states to the preset number is greater than a third preset threshold, determining that the first state is the real state of the target.

[0130] In the embodiments of the present disclosure, if the third proportion of the number of states in the second state that are the same as the first state to the preset number is greater than the third preset threshold, the vehicle-mounted terminal can determine that the first state is the real state of the target. The third preset threshold can be set according to needs, for example, the third preset threshold can be 0.5, which will not be specifically limited here.

[0131] Therefore, the results of determining the state of the target in each road image based on the target pixels of the target contained in the target tracking frame and the other pixels of other targets other than the target, and the position of the target tracking frame in the road image in multiple road images can be voted to determine the authenticity of the target state, thereby improving the stability and accuracy of target state recognition.

[0132] Figure 7 FIG. 7 is a structural schematic diagram of a target state recognition device provided by an embodiment of the present disclosure. The device can be understood as the vehicle-mounted terminal or part of the functional modules in the vehicle-mounted terminal. As shown in FIG. 7, the target state recognition device 700 can include: Figure 7

[0133] A first obtaining module 710 is configured to obtain a road image in the driving direction of a target vehicle.

[0134] A second obtaining module 720 is configured to obtain, based on the road image, a target tracking frame of the target in the road image, and target pixels corresponding to the target contained in the target tracking frame and other pixels corresponding to other targets other than the target contained in the target tracking frame, the target including pedestrians and cyclists.

[0135] A first determining module 730 is configured to compare the target pixels contained in the target tracking frame with the other pixels contained in the target tracking frame to determine the state of the target.

[0136] Optionally, the first determining module 730 can include:

[0137] A first statistical sub-module is configured to count the number of target pixels and the number of other pixels contained in the target tracking frame.

[0138] A first calculation sub-module is configured to calculate a first ratio between the number of other pixels and the number of target pixels.

[0139] A first determination sub-module is configured to determine that the state of the target is that the target is occluded by other targets if the first ratio is greater than a first preset threshold.

[0140] Optionally, the first determining module 730 can include:

[0141] A second statistical sub-module is configured to count the number of target pixels contained in the target tracking frame and the number of pixels in the road driving area.

[0142] A second calculation sub-module is configured to calculate a second ratio between the number of pixels in the road driving area and the number of target pixels.

[0143] ​The second determining sub-module is configured to determine that the state of the target is that the target is in a road driving area if the second ratio is greater than a second preset threshold.

[0144] Optionally, the target state identification device 700 can include:

[0145] The second determining module is configured to determine the state of the target based on a position of the target tracking frame in the road image.

[0146] Optionally, the second determining module can include:

[0147] The third determining sub-module is configured to determine a position of a lower frame line midpoint pixel of the target tracking frame in the road image.

[0148] The fourth determining sub-module is configured to determine the state of the target based on the position of the lower frame line midpoint pixel in the road image.

[0149] Optionally, the fourth determining sub-module can include:

[0150] The first determining sub-unit is configured to determine that the state of the target is that the target is in a close distance to the target vehicle if the position of the lower frame line midpoint pixel is in a first preset pixel area at the bottom of the road image.

[0151] The second determining sub-unit is configured to determine that the state of the target is that the target is in a far distance to the target vehicle if a distance between the position of the lower frame line midpoint pixel and a vanishing pixel point in the road image is less than a preset distance threshold.

[0152] The third determining sub-unit is configured to determine that the state of the target is that the target cuts into the picture if the position of the lower frame line midpoint pixel is in a second preset pixel area at a lower left corner or a lower right corner of the road image.

[0153] Optionally, the target tracking frame includes at least one of a pedestrian tracking frame, a cyclist tracking frame, and a cyclist fusion frame, and the cyclist fusion frame includes the pedestrian tracking frame and the cyclist tracking frame.

[0154] Optionally, the target state identification device 700 can include:

[0155] The third acquiring module is configured to acquire a vertical distance between the pedestrian tracking frame and the cyclist tracking frame when both the pedestrian tracking frame and the cyclist tracking frame exist in the target tracking frame.

[0156] The third determining module is configured to determine the cyclist tracking frame as the adjusted target tracking frame if the vertical distance is greater than a preset distance threshold.

[0157] The fusion module is configured to fuse the pedestrian tracking frame and the cyclist tracking frame to generate a cyclist fusion frame if the longitudinal distance is less than or equal to a preset distance threshold.

[0158] The fourth determination module is configured to determine the cyclist fusion frame as the adjusted target tracking frame.

[0159] The fifth determination module is configured to compare a target pixel included in the adjusted target tracking frame with other pixels included in the adjusted target tracking frame to determine a state of the target.

[0160] Optionally, the target state recognition device 700 can include:

[0161] The sixth determination module is configured to determine the state of the target as the first state.

[0162] The third acquisition module is configured to acquire, for each first state, a preset number of frames of road images corresponding to the first state.

[0163] The seventh determination module is configured to compare a target pixel included in a target tracking frame in each road image with other pixels included in the target tracking frame to determine a second state of the target in each road image.

[0164] The statistical module is configured to count a number of states that are the same as the first state in the second state.

[0165] The eighth determination module is configured to determine that the first state is a real state of the target if a third proportion of the number of states to the preset number is greater than a third preset threshold.

[0166] The device provided by the embodiments of the present disclosure can implement the method of any of the above-mentioned embodiments, and has similar implementation manners and beneficial effects, which will not be described here again.

[0167] The embodiments of the present disclosure also provide a computer device including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method of any of the above-mentioned embodiments can be implemented, and has similar implementation manners and beneficial effects, which will not be described here again.

[0168] Figure 8 is a structural schematic diagram of a vehicle terminal provided by the embodiments of the present disclosure, as Figure 8 shown, the vehicle terminal 800 can include a processor 810 and a memory 820, wherein the memory 820 stores a computer program 821, and when the computer program 821 is executed by the processor 810, the method provided by any of the above-mentioned embodiments can be implemented, and has similar implementation manners and beneficial effects, which will not be described here again.

[0169] Of course, in order to simplify, Figure 8Only some of the components of the in-vehicle terminal 800 related to the present application are shown, and components such as a bus, an input / output interface, an input device, and an output device are omitted. In addition to these, the in-vehicle terminal 800 can include any other appropriate components according to the specific application.

[0170] The embodiment of the present disclosure provides a computer readable storage medium, the storage medium stores a computer program, when the computer program is executed by a processor, the method of any one of the above embodiments can be implemented, the execution mode and beneficial effects are similar, and here will not be repeated.

[0171] The computer readable storage medium described above can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0172] The computer program described above can be written in any combination of one or more programming languages for executing the operations of the embodiments of the present disclosure, and the programming languages include object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The program code can be executed entirely on the user in-vehicle terminal, partially on the user device, as a separate software package, partially on the user in-vehicle terminal and partially on the remote in-vehicle terminal, or entirely on the remote in-vehicle terminal or server.

[0173] The embodiment of the present disclosure also provides a vehicle, which includes the in-vehicle terminal described above, and the method of any one of the above embodiments can be implemented, the execution mode and beneficial effects are similar, and here will not be repeated.

[0174] The above is only a specific embodiment of the present disclosure, which enables those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A target state recognition method characterized by comprising: The method comprises the following steps: acquiring a road image of a driving direction of a target vehicle; based on the road image, acquiring a target tracking frame of a target in the road image, and target pixels corresponding to the target contained in the target tracking frame and other pixels corresponding to other targets other than the target contained in the target tracking frame, the target including a pedestrian and a cyclist, the target tracking frame including at least one of a pedestrian tracking frame, a cyclist tracking frame, and a cyclist fusion frame, the cyclist fusion frame including a pedestrian tracking frame and a cyclist tracking frame; comparing the target pixels contained in the target tracking frame with the other pixels contained in the target tracking frame to determine a state of the target.

2. The method of claim 1, wherein, The step of comparing the target pixels contained in the target tracking frame with the other pixels contained in the target tracking frame to determine the state of the target comprises the following steps: counting a number of the target pixels and a number of the other pixels; calculating a first ratio between the number of the other pixels and the number of the target pixels; if the first ratio is greater than a first preset threshold, determining that the state of the target is that the target is occluded by the other target.

3. The method of claim 1, wherein, The other target includes a road driving area, and the step of comparing the target pixels contained in the target tracking frame with the other pixels contained in the target tracking frame to determine the state of the target comprises the following steps: counting a number of the target pixels and a number of pixels of the road driving area; calculating a second ratio between the number of the pixels of the road driving area and the number of the target pixels; if the second ratio is greater than a second preset threshold, determining that the state of the target is that the target is in the road driving area.

4. The method of claim 1, wherein, After the step of acquiring the target tracking frame of the target in the road image based on the road image, the method further comprises the following steps: determining the state of the target based on a position of the target tracking frame in the road image.

5. The method of claim 4, wherein, The step of determining the state of the target based on the position of the target tracking frame in the road image comprises the following steps: determining a position of a lower frame line midpoint pixel of the target tracking frame in the road image; determining the state of the target based on the position of the lower frame line midpoint pixel in the road image.

6. The method of claim 5, wherein, The step of determining the state of the target based on the position of the lower frame line midpoint pixel in the road image comprises the following steps: if the position of the lower frame line midpoint pixel is in a first preset pixel area at a bottom of the road image, determining that the state of the target is that the target is at a close distance from the target vehicle; if a distance between the position of the lower frame line midpoint pixel and a disappearing pixel point in the road image is less than a preset distance threshold, determining that the state of the target is that the target is at a far distance from the target vehicle; if the position of the lower frame line midpoint pixel is in a second preset pixel area at a lower left corner or a lower right corner of the road image, determining that the state of the target is that the target cuts into the picture.

7. The method of claim 1, wherein, After the target tracking box of the target in the road image is obtained based on the road image, the method further includes: When the pedestrian tracking box and the cyclist tracking box exist in the target tracking box, a longitudinal distance between the pedestrian tracking box and the cyclist tracking box is obtained; If the longitudinal distance is greater than a preset distance threshold, the cyclist tracking box is determined as an adjusted target tracking box; If the longitudinal distance is less than or equal to the preset distance threshold, the pedestrian tracking box and the cyclist tracking box are fused to generate a cyclist fusion box; The cyclist fusion box is determined as the adjusted target tracking box; The target pixel included in the target tracking box is compared with the other pixel included in the target tracking box to determine the state of the target.

8. The method of claim 1, wherein, After the target pixel included in the target tracking box is compared with the other pixel included in the target tracking box to determine the state of the target, the method further includes: The state of the target is determined as a first state; For each first state, a preset number of frames of road images corresponding to the first state are obtained; The target pixel included in the target tracking box in each road image is compared with the other pixel included in the target tracking box to determine a second state of the target in each road image; The number of states in the second state that are the same as the first state is counted; If a third proportion of the number of states to the preset number is greater than a third preset threshold, the first state is determined as a real state of the target.

9. A target state recognition apparatus characterized by comprising: Comprise: The first acquisition module is used for acquiring a road image of a target vehicle driving direction; The second acquisition module is used for acquiring a target tracking box of a target in the road image based on the road image, and target pixels corresponding to the target and other pixels corresponding to other targets other than the target included in the target tracking box, the target including pedestrians and cyclists, the target tracking box including at least one of a pedestrian tracking box, a cyclist tracking box and a cyclist fusion box, the cyclist fusion box including a pedestrian tracking box and a cyclist tracking box; The first determination module is used for comparing the target pixel included in the target tracking box with the other pixel included in the target tracking box to determine the state of the target.

10. A vehicle terminal, characterized by comprising: Comprise: A memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the target state identification method in any one of claims 1-8 is implemented.

11. A computer readable storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by the processor, the target state identification method in any one of claims 1-8 is implemented.

12. A vehicle characterized by comprising: The vehicle comprises the vehicle terminal in claim 10.

Images