Line pressing detection method and device, vehicle and storage medium

Abstract

The invention discloses a line pressing detection method and device, a vehicle and a readable storage medium. The method comprises the steps that a target image area where a target vehicle is located is determined from a shot image of a vehicle; the target vehicle is a vehicle around the vehicle; performing fusion processing on the target image area and the regulation and control response data of the vehicle to obtain a fusion feature corresponding to the target vehicle; the regulation control response data of the vehicle is used for indicating the change of the motion state of the vehicle; performing line pressing detection based on the fusion features to obtain a line pressing detection result corresponding to the target vehicle; the line pressing detection result indicates whether the target vehicle presses the lane line of the vehicle lane where the vehicle is located. According to the method provided by the invention, the accuracy of the line pressing detection result of the target vehicle is improved, so that the driving safety based on the line pressing detection result of the target vehicle is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and more specifically, to a wire pressing detection method, device, vehicle and computer-readable storage medium. Background Art

[0002] In the field of autonomous driving, in order to ensure that the autonomous vehicle does not violate traffic regulations and drives safely and comfortably, it is usually necessary to judge whether the vehicles around the vehicle cross the line. Currently, it is possible to analyze whether the vehicles around the vehicle cross the line through the images collected by the vehicle.

[0003] However, when the aforementioned line crossing detection method is used for line crossing detection, the accuracy of line crossing detection is low, and it is difficult to accurately determine whether vehicles around the vehicle cross the line, resulting in poor safety when driving based on the line crossing detection result. Summary of the invention

[0004] The present application proposes a wire crossing detection method, device, vehicle and computer-readable storage medium to improve the accuracy of wire crossing detection and thereby improve the safety of vehicle driving.

[0005] In a first aspect, an embodiment of the present application provides a wire pressing detection method, the method comprising:

[0006] Determine a target image area where a target vehicle is located from the captured image of the vehicle; the target vehicle is a vehicle located around the vehicle;

[0007] The target image area and the control response data of the ego vehicle are fused to obtain the fusion features corresponding to the target vehicle; the control response data of the ego vehicle is used to indicate the change of the motion state of the ego vehicle;

[0008] Based on the fusion features, line crossing detection is performed to obtain the line crossing detection result corresponding to the target vehicle; the line crossing detection result indicates whether the target vehicle crosses the lane line of the own vehicle lane where the own vehicle is located.

[0009] In a second aspect, an embodiment of the present application further provides a wire pressing detection device, the device comprising:

[0010] A determination module is used to determine a target image area where a target vehicle is located from the captured image of the vehicle; the target vehicle is a vehicle located around the vehicle;

[0011] A fusion module is used to fuse the target image area and the control response data of the ego vehicle to obtain the fusion features corresponding to the target vehicle; the control response data of the ego vehicle is used to indicate the change of the motion state of the ego vehicle;

[0012] The detection module is used to perform lane crossing detection based on the fusion features to obtain a lane crossing detection result corresponding to the target vehicle; the lane crossing detection result indicates whether the target vehicle crosses the lane line of the vehicle lane where the vehicle is located.

[0013] In a third aspect, an embodiment of the present application further provides a vehicle, characterized in that the vehicle comprises: one or more processors; a memory; one or more applications, wherein one or more applications are stored in the memory and configured to be executed by one or more processors, and one or more programs are configured to execute the above method.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the above method.

[0015] The present application provides a line crossing detection method, device, vehicle and computer-readable storage medium. In the present application, a target image area where a target vehicle is located is determined from a captured image of an own vehicle, and then the target image area and the regulatory control response data of the own vehicle are fused to obtain a fusion feature corresponding to the target vehicle, and then based on the fusion feature, a line crossing detection result corresponding to the target vehicle is determined. The regulatory control response data of the own vehicle more directly reflects the impact of the driving state of the target vehicle on the own vehicle, so that the regulatory control response data of the own vehicle also reflects whether the target vehicle crosses the line. Therefore, the target fusion feature that fuses the target image area where the target vehicle is located and the regulatory control response data of the own vehicle more accurately indicates whether the target vehicle crosses the line, so that the line crossing detection result determined based on the target fusion feature is more accurate, thereby improving the safety when driving based on the line crossing detection result.

[0016] Other features and advantages of the embodiments of the present application will be described in the subsequent description, and partly become apparent from the description, or can be understood by practicing the embodiments of the present application. The purposes and other advantages of the embodiments of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A schematic diagram of a vehicle hardware environment suitable for an embodiment of the present application is shown.

[0019] Figure 2 A flow chart of a wire pressing detection method proposed according to an embodiment of the present application is shown.

[0020] Figure 3 Show Figure 2 The corresponding step S110 of the embodiment is a flowchart in an embodiment.

[0021] Figure 4 A schematic diagram showing a process of determining a target image area in an embodiment of the present application is shown.

[0022] Figure 5 A schematic diagram of a wire pressing detection process in an embodiment of the present application is shown.

[0023] Figure 6 A structural block diagram of a wire pressing detection device proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In order to make those skilled in the art better understand the present application scheme, the technical scheme in the present application embodiment will be clearly and completely described below in conjunction with the drawings in the present application embodiment. Obviously, the described embodiment is only a part of the present application embodiment, rather than all the embodiments. The components of the present application embodiment usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents the selected embodiment of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0025] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0026] Reference Figure 1 , Figure 1 A schematic diagram of a vehicle hardware environment suitable for an embodiment of the present application is shown, wherein the vehicle 100 includes a driving system 110, which may have multiple built-in autonomous driving functions. The driving system 110 may store an electronic map, and the driving system 110 may plan a driving route based on the electronic map stored in the driving system, and may also control the vehicle's autonomous driving based on the planned driving route.

[0027] The driving system 110 may include a data acquisition device 111 , one or more (only one is shown in the figure) processors 112 , and a memory 113 .

[0028] The data acquisition device 111 is used to detect the position information of the vehicle and the environmental information around the vehicle. The data acquisition device 111 may include a camera, a vehicle speed sensor, a steering wheel angle sensor, etc. The camera is used to capture the environmental information around the vehicle to obtain a captured image. The environmental information may include lanes, vehicles around the vehicle, and obstacles around the vehicle.

[0029] The processor 112 may be a microcontroller unit (MCU) having a built-in memory 113 . The memory 113 stores a program that can execute the contents of the following embodiments, and the processor 112 may execute the program stored in the memory 113 .

[0030] The processor 112 may include one or more processors. The processor 112 uses various interfaces and lines to connect various parts of the entire vehicle 100, and executes various functions of the vehicle 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 113, and calling data stored in the memory 113.

[0031] The memory 113 may include a random access memory (RAM) or a read-only memory (ROM). The memory 15 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 15 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.

[0032] See also Figure 2 , Figure 2 A flow chart of a wire pressing detection method proposed according to an embodiment of the present application is shown, which is used for a vehicle, and the method includes:

[0033] S110: Determine a target image area where a target vehicle is located from the captured image of the own vehicle.

[0034] Among them, the target vehicle is the vehicle located around the ego vehicle; for example, the target vehicle may refer to all vehicles located around the ego vehicle, or the target vehicle may refer to a vehicle located in an adjacent lane adjacent to the ego vehicle lane where the ego vehicle is located, or the target vehicle may refer to a vehicle located in an adjacent lane and closest to the ego vehicle in front of the ego vehicle.

[0035] The vehicle in this embodiment can be an electric vehicle or a fuel vehicle, or a sedan, an SUV, a bus, a truck, etc.; the self-vehicle refers to the vehicle itself.

[0036] The vehicle may have a camera for collecting images of the environment around the vehicle, and the image collected by the camera is used as a captured image. The pixel area where the target vehicle is located can be determined from the captured image as the target image area.

[0037] In the present application, a vehicle recognition model can be used to perform vehicle recognition on the captured image to obtain a target image area of ​​the target vehicle, wherein the vehicle recognition model can be a convolutional neural network.

[0038] First, a first sample image and a sample annotation corresponding to the first sample image may be obtained, and then a neural network model with parameter initialization may be trained by the first sample image and the sample annotation corresponding to the first sample image to obtain a vehicle recognition model. The first sample image may be an image taken by a camera of the first sample vehicle (which may be the vehicle itself or another vehicle) during driving, and the first sample image may include other vehicles other than the first sample vehicle, and the sample annotation is used to indicate a pixel area in the first sample image where other vehicles other than the first sample vehicle are located.

[0039] The first sample image can be used to perform vehicle recognition through an initial vehicle recognition model initialized with parameters to obtain a predicted pixel area, wherein the predicted pixel area indicates a pixel area in the first sample image where other vehicles except the first sample vehicle are located, and then based on the difference between the predicted pixel area and the sample annotation, the initial vehicle recognition model initialized with parameters is trained to obtain a vehicle recognition model.

[0040] In some embodiments, the predicted pixel area can be in the form of a rectangular box, which may include the coordinates of key points (center points, corner points, etc.) in the rectangular box (pixel coordinates in the first sample image) and an indication of the size of the rectangular box. Accordingly, the sample is marked as a rectangular box that selects other vehicles other than the first sample vehicle in the first sample image.

[0041] In the present application, in order to improve the recognition effect of the vehicle recognition model, the first sample image may include images taken under different lighting, different roads, different traffic densities, different lane line types / colors / positions / postures, different vehicle types / colors / sizes / positions / postures, different vehicle speeds, different weather and other environments.

[0042] S120, fusing the target image area and the control response data of the vehicle to obtain a fusion feature corresponding to the target vehicle.

[0043] The control response data of the vehicle is used to indicate the change of the motion state of the vehicle. For example, the control response data of the vehicle may include the acceleration, jerk and steering wheel angular velocity of the vehicle. The acceleration of the vehicle may include the longitudinal acceleration ax and the lateral phase velocity ay of the vehicle in the driving direction, the jerk of the vehicle may include the longitudinal jerk jx and the lateral jerk jy of the vehicle in the driving direction, and the steering wheel angular velocity of the vehicle may be represented by yaw.

[0044] For the judgment of whether the target lane is crossing the line, the control response data of the ego vehicle is a more direct information feedback. For example, if the target vehicle is in a state of not crossing the line, the movement state of the ego vehicle does not change significantly; if the target vehicle is in an expected crossing line (expected crossing line refers to the vehicle behavior that will inevitably cross the line, for example, the vehicle changing lanes and expected crossing line), the ego vehicle will swerve the steering wheel and take a detour; if the target vehicle is in an unexpected crossing line (unexpected crossing line is a vehicle behavior that may not necessarily cause crossing line, for example, the vehicle suddenly cuts in), the ego vehicle will brake in response. Therefore, when the target vehicle is in a state of not crossing the line: all control response data tend to 0; when the target vehicle is in an expected crossing line state: ay, jy, yaw all take large values; when the target vehicle is in an unexpected crossing line state: ax and jx all take large values.

[0045] In some embodiments, feature extraction can be performed on the target image area through a feature extraction model to obtain target image features, and then the target image features and the regulatory control response data of the own vehicle are summed, multiplied, or concatenated to achieve fusion of the target image features and the regulatory control response data of the own vehicle to obtain fused features corresponding to the target vehicle.

[0046] The control response vector Y = [ax.ay, jx, jy, yaw] can be constructed through the control response data, and then the control response vector and the target image features are summed, multiplied or concatenated to achieve the fusion of the target image features and the control response data of the vehicle to obtain the fusion features corresponding to the target vehicle.

[0047] In some other embodiments, feature extraction can be performed on the target image area through a feature extraction model to obtain target image features; the target image features and the control response data of the vehicle are fused through weight parameters to obtain fused features corresponding to the target vehicle.

[0048] The weight parameter may be a fixed value set based on demand, or may be an adjusted weight parameter obtained by adjusting a preset weight parameter.

[0049] Specifically, the product of the weight parameter target image feature and the control response data of the vehicle can be calculated, and the obtained product result is used as the fusion feature corresponding to the target vehicle. At this time, the determination process of the fusion feature corresponding to the target vehicle can be briefly described as the following formula:

[0050] Feature=W*Y*X

[0051] Among them, Feature is the fusion feature corresponding to the target vehicle, W is the weight parameter, Y is the regulatory response vector constructed based on the control response data.

[0052] Among them, the feature extraction model can be used for the neural network model. The second sample image taken during the driving of the second sample vehicle (which can be the same as the first sample vehicle or different from the first sample vehicle) and the line-pressing annotation information corresponding to the reference sample vehicle (other vehicles in the second sample image excluding the second sample vehicle) can be obtained. The line-pressing annotation information indicates whether the reference sample vehicle presses the lane line of the lane where the second sample vehicle is located. For example, the line-pressing annotation information may include the reference sample vehicle pressing the line and not pressing the line (wherein, not pressing the line can also be subdivided into expected pressing and unexpected pressing). The second sample image can also be a pixel area including the reference sample vehicle determined from the captured image taken by the second sample vehicle, that is, the second sample image can also be a part of the captured image taken from the second sample vehicle.

[0053] Then, the initial feature extraction model and the initial classifier with parameter initialization are trained through the second sample image, the regulatory control response data of the second sample vehicle and the second sample image, and the trained initial feature extraction model is used as the feature extraction model, and the trained initial classifier is used as the line crossing detection model, so as to determine the fusion features based on the target vehicle through the line crossing detection model and determine the line crossing probability corresponding to the target vehicle.

[0054] During the training process, the initial feature extraction model is used to extract the features of the second sample image, and then the features extracted by the initial feature extraction model and the regulatory response data of the second sample vehicle are fused through weight parameters to obtain sample fusion features, and then the sample fusion features are classified and processed by the initial classifier to obtain the predicted sample line-crossing probability of each reference sample vehicle, and then based on the difference between the sample line-crossing probability and the line-crossing annotation information, the initial feature extraction model and the initial classifier with initialized parameters are trained.

[0055] Among them, the predicted sample crossing the line probability output by the initial classifier can be in the form of probability. The higher the probability, the higher the possibility that the reference sample vehicle crosses the line. Correspondingly, the crossing line marking information can be in the form of a value of 0 or 1, 0 indicates that the reference sample vehicle crosses the line, and 1 indicates that the reference sample vehicle does not cross the line.

[0056] In some embodiments, the initial feature extraction model, initial classifier and preset weight parameters with parameter initialization can be trained (that is, the specific values ​​of the weight parameters are adjusted) through the second sample image, the regulatory response data of the second sample vehicle and the sample crossing probability determined by the second sample image, so as to realize the training of the initial feature extraction model and the initial classifier, and realize the adjustment of the weight parameters at the same time, so that the weight parameters can be more accurate.

[0057] In the present application, in order to improve the recognition effect of the feature extraction model and the cross-line detection model, the second sample image may include images taken under different lighting, different roads, different traffic densities, different lane line types / colors / positions / postures, different vehicle types / colors / sizes / positions / postures, different vehicle speeds, different weather and other environments.

[0058] S130: Perform line crossing detection based on the fusion features to obtain a line crossing detection result corresponding to the target vehicle.

[0059] The line crossing detection result indicates whether the target vehicle crosses the lane line of the vehicle lane where the vehicle is located. In this embodiment, the line crossing detection can be performed based on the fusion features using the aforementioned line crossing detection model to obtain the line crossing detection result corresponding to the target vehicle.

[0060] As an implementation method, the line-crossing detection result of the target vehicle may be in the form of probability. The higher the probability, the higher the possibility that the target vehicle crosses the line, and the lower the probability, the lower the possibility that the target vehicle crosses the line. At this time, the line-crossing detection model outputs a predicted probability based on the fusion feature. If the predicted probability is greater than the probability threshold, the line-crossing detection result of the target vehicle crossing the lane line of the own vehicle lane where the own vehicle is located is obtained. If the predicted probability is not greater than the probability threshold, the line-crossing detection result of the target vehicle crossing the lane line of the own vehicle lane where the own vehicle is located is obtained. The probability threshold can be set based on demand, such as 0.7.

[0061] It is worth mentioning that there may be multiple target vehicles. For each target vehicle, the respective line crossing detection result is determined according to the aforementioned S110-S130, which will not be repeated here.

[0062] In this embodiment, a target image area where the target vehicle is located is determined from the captured image of the own vehicle, and then the target image area and the regulatory control response data of the own vehicle are fused to obtain a fusion feature corresponding to the target vehicle, and then based on the fusion feature, a line crossing detection result corresponding to the target vehicle is determined. The regulatory control response data of the own vehicle more directly reflects the impact of the driving state of the target vehicle on the own vehicle, so that the regulatory control response data of the own vehicle also reflects whether the target vehicle crosses the line. Therefore, the target fusion feature that fuses the target image area where the target vehicle is located and the regulatory control response data of the own vehicle more accurately indicates whether the target vehicle crosses the line, so that the line crossing detection result determined based on the target fusion feature is more accurate, thereby improving the safety of driving based on the line crossing detection result.

[0063] In one embodiment, if Figure 3 As shown, S110 also includes:

[0064] S210: Based on the captured image, determine the lane line position of the lane of the own vehicle in the captured image and the target vehicle area where the target vehicle is located in the captured image.

[0065] In this embodiment, the captured image can be used to identify the vehicle through a vehicle recognition model to obtain a target vehicle area of ​​the target vehicle in the captured image; the vehicle recognition model is constructed based on a convolutional neural network; the target image is used to identify the lane line of the own vehicle through a lane line recognition model to obtain the lane line position of the lane of the own vehicle in the captured image; the lane line recognition model is constructed based on a convolutional neural network.

[0066] The training process of the vehicle recognition model is as described in the above embodiment, which will not be repeated here. The vehicle recognition model uses the pixel area predicted by the target image as the target vehicle area of ​​the target vehicle in the captured image.

[0067] A third sample image taken during the driving of a third sample vehicle (which may be the same as or different from the first sample vehicle) and a sample lane line position of the lane where the third sample vehicle is located marked in the second sample image can be obtained, and then the initial lane line recognition model with initialized parameters is trained using the third sample image and the sample lane line to obtain a lane line recognition model.

[0068] The third sample image can be input into the parameter-initialized initial lane line recognition model to obtain the predicted sample lane line position predicted by the parameter-initialized initial lane line recognition model, and then the parameter-initialized initial lane line recognition model is trained based on the difference between the predicted sample lane line position and the sample lane line position.

[0069] The lane line recognition model directly performs lane line recognition based on the captured image of the ego vehicle, and predicts the lane line position of the ego vehicle's lane in the captured image.

[0070] The lane line positions in this embodiment (the lane line positions of the vehicle lane involved and the sample lane line positions of the lane where the third sample vehicle is located) can be in the form of line segments or point sets, and this application does not limit this.

[0071] In the present application, in order to improve the recognition effect of the lane line recognition model, the third sample image may include images taken under different lighting, different roads, different traffic densities, different lane line types / colors / positions / postures, different vehicle types / colors / sizes / positions / postures, different vehicle speeds, different weather and other environments.

[0072] S220: Based on the target vehicle area and the lane line position, determine the target image area where the target vehicle is located from the captured image of the own vehicle.

[0073] After the target vehicle region and the lane line position are obtained, a pixel region including the target vehicle region in the captured image is determined as the target image region in combination with the relative position relationship between the target vehicle region and the lane line position.

[0074] In some implementations, based on the relative position relationship between the target vehicle region and the lane line position, the target lane line closest to the target vehicle region can be selected from the lane lines of the vehicle lane; based on the relative position relationship between the target lane line and the target vehicle region, the target vehicle region is expanded to obtain the target image region where the target vehicle is located. The lane lines of the vehicle lane include left lane lines and right lane lines.

[0075] Exemplarily, when the target vehicle area is in the form of a rectangular frame, the target vehicle area can be indicated by the coordinates of the center point of the target vehicle area in the captured image as (pos_x, pos_y) and the size of the target vehicle area (size_x, size_y). At this time, the left lane line of the lane where the ego vehicle is located is represented by line_left, and the right lane line of the lane where the ego vehicle is located is represented by line_right. The distances between the center point (pos_x, pos_y) of the target vehicle area and line_left and line_right, respectively, can be determined as the distances between the target vehicle area and line_left and line_right, respectively, and then one of the distances is selected as the target lane line corresponding to the target vehicle.

[0076] Then, based on the relative position relationship between the target lane line and the target vehicle area, the target vehicle area is expanded to obtain a target image area including the target vehicle area.

[0077] Optionally, the target image area where the target vehicle is located can be obtained by expanding the first size in the direction close to the target lane line, the second size in the driving direction of the target vehicle, and the third size in the direction opposite to the driving direction of the target vehicle with the target vehicle area as the center; the first size is determined based on the lane width of the vehicle lane. Exemplarily, the first size, the second size, and the third size can be set based on demand, for example, the first size can be the lane width of the vehicle lane or 0.7 of the lane width of the vehicle lane, and the second size and the third size are both 0.5m.

[0078] It can be understood that the expansion size (first size, second size and third size) can be selected according to the performance of the feature extraction model, the performance of the line crossing detection model and the amount of training data used to train them (including the second sample image, the regulatory control response data of the second sample vehicle and the sample line crossing probability determined by the second sample image, etc.). The larger the expansion size, the more information in the image area that needs to be analyzed, the longer the training time, and the more training resources are required. Similarly, the larger the amount of training data used to train them, the longer the training time, and the more training resources are required.

[0079] For example, Figure 4 As shown, the left lane line of the own vehicle lane is 401, the right lane line of the own vehicle lane is 402, the determined target vehicle area is 403, the target vehicle area 403 is close to the right lane line 402, the right lane line 402 is determined to be the target lane line, and then the target vehicle area 403 is taken as the center, the first size b is expanded in the direction close to the right lane line (that is, the left side of the target vehicle area 403), the second size a is expanded in the driving direction of the vehicle, and the third size c is expanded in the direction opposite to the driving direction of the vehicle, and the expanded area 404 is obtained as the target image area.

[0080] It is worth mentioning that the second sample image used in the aforementioned training process may also be an image region determined for the reference sample vehicle from the captured image of the second sample vehicle in the manner of S210-S220. At this time, a second sample image is determined for each reference sample vehicle, so that the predicted sample crossing probability of each reference sample vehicle is determined based on the second sample image of each reference sample vehicle and the regulatory response data of the second sample vehicle.

[0081] It is understandable that the target image area determined for the target vehicle is actually a part of the captured image of the own vehicle, that is, it is necessary to extract the region of interest (ROI) of the target vehicle from the captured image of the own vehicle to obtain the target image area of ​​the target vehicle.

[0082] For example, in this embodiment, Figure 5 As shown, firstly, the captured image of the ego vehicle is acquired, then vehicle recognition is performed to determine the target vehicle area, and lane line recognition is performed to determine the lane line position of the ego vehicle lane, and then the target vehicle area and the lane line position of the ego vehicle lane are combined to expand the target vehicle area of ​​the target vehicle to determine the target image area of ​​the target vehicle, and then the features of the target image area are extracted through the feature extraction model to obtain the target image features, and then the target image features and the control response data of the ego vehicle are fused to obtain the fused features, and then the line pressing detection is performed based on the fused features to obtain the line pressing detection result of the target vehicle.

[0083] In this embodiment, the target vehicle area is adaptively expanded through the target vehicle area where the target vehicle is located, so as to extract the target image area where the target vehicle is located from the captured image, and perform cross-line detection directly based on the target image area, thereby eliminating the need to analyze the entire captured image, greatly reducing the amount of data, and improving the detection efficiency of cross-line detection.

[0084] See attached Figure 6 , Figure 6 The structural block diagram of a wire pressing detection device proposed in one embodiment of the present application is shown. For vehicles, the device 800 includes:

[0085] The determination module 810 is used to determine the target image area where the target vehicle is located from the captured image of the vehicle; the target vehicle is a vehicle located around the vehicle;

[0086] The fusion module 820 is used to fuse the target image area and the control response data of the ego vehicle to obtain a fusion feature corresponding to the target vehicle; the control response data of the ego vehicle is used to indicate the change of the motion state of the ego vehicle;

[0087] The detection module 830 is used to perform lane crossing detection based on the fusion features to obtain a lane crossing detection result corresponding to the target vehicle; the lane crossing detection result indicates whether the target vehicle crosses the lane line of the vehicle lane where the vehicle is located.

[0088] Optionally, the determination module 810 is also used to determine, based on the captured image, the lane line position of the lane of the own vehicle in the captured image and the target vehicle area where the target vehicle is located in the captured image; based on the target vehicle area and the lane line position, determine the target image area where the target vehicle is located from the captured image.

[0089] Optionally, the determination module 810 is also used to select a target lane line that is closest to the target vehicle area from the lane lines of the own vehicle lane based on the relative position relationship between the target vehicle area and the lane line position; and to perform regional expansion on the target vehicle area based on the relative position relationship between the target lane line and the target vehicle area to obtain a target image area where the target vehicle is located.

[0090] Optionally, the determination module 810 is also used to expand a first size in a direction close to the target lane line, expand a second size in the driving direction of the target vehicle, and expand a third size in a direction opposite to the driving direction of the target vehicle with the target vehicle area as the center, so as to obtain a target image area where the target vehicle is located; the first size is determined based on the lane width of the own vehicle lane.

[0091] Optionally, the determination module 810 is also used to perform vehicle identification on the captured image through a vehicle identification model to obtain a target vehicle area of ​​the target vehicle in the captured image; the vehicle identification model is constructed based on a convolutional neural network; the lane line of the own vehicle is identified on the target image through a lane line recognition model to obtain the lane line position of the lane of the own vehicle in the captured image; the lane line recognition model is constructed based on a convolutional neural network.

[0092] Optionally, the fusion module 820 is also used to extract features from the target image area through a feature extraction model to obtain target image features; and to fuse the target image features and the control response data of the vehicle through weight parameters to obtain fusion features corresponding to the target vehicle.

[0093] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0094] In addition, each function in each embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of software function module.

[0095] In addition, each function in each embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of software function module.

[0096] In addition, each function in each embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of software function module.

[0097] On the other hand, the present application also provides a computer-readable storage medium, in which program code is stored. The program code can be called by a processor to execute the method described in the above method embodiment.

[0098] The computer readable storage medium may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a cluster of ROMs. Optionally, the computer readable storage medium comprises a non-transitory computer-readable storage medium. The computer readable storage medium has storage space for program codes that execute any of the method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code can be compressed, for example, in an appropriate form.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A wire pressing detection method, characterized in that: The method comprises: Determine a target image area where a target vehicle is located from a captured image of the vehicle; the target vehicle is a vehicle located around the vehicle; The target image area and the control response data of the ego vehicle are fused to obtain a fusion feature corresponding to the target vehicle; the control response data of the ego vehicle is used to indicate a change in the motion state of the ego vehicle; A lane crossing detection is performed based on the fusion feature to obtain a lane crossing detection result corresponding to the target vehicle; the lane crossing detection result indicates whether the target vehicle crosses the lane line of the own vehicle lane where the own vehicle is located.

2. The method according to claim 1, characterized in that The step of determining a target image area where a target vehicle is located from the captured image of the vehicle includes: Based on the captured image, determining a lane line position of the lane of the vehicle in the captured image and a target vehicle area where the target vehicle is located in the captured image; Based on the target vehicle area and the lane line position, a target image area where the target vehicle is located is determined from the captured image.

3. The method according to claim 2, characterized in that The step of determining a target image area where the target vehicle is located from the captured image based on the target vehicle area and the lane line position includes: Based on the relative position relationship between the target vehicle area and the lane line position, selecting a target lane line closest to the target vehicle area from the lane lines of the vehicle lane; Based on the relative positional relationship between the target lane line and the target vehicle region, the target vehicle region is expanded to obtain a target image region where the target vehicle is located.

4. The method according to claim 3, characterized in that The method of performing regional expansion on the target vehicle region based on the relative position relationship between the target lane line and the target vehicle region to obtain a target image region where the target vehicle is located includes: Taking the target vehicle area as the center, the first size is expanded in the direction close to the target lane line, the second size is expanded in the driving direction of the target vehicle, and the third size is expanded in the direction opposite to the driving direction of the target vehicle, so as to obtain the target image area where the target vehicle is located; the first size is determined based on the lane width of the own vehicle lane.

5. The method according to claim 2, characterized in that: The determining, based on the captured image, a lane line position of the lane of the vehicle in the captured image and a target vehicle region where the target vehicle is located in the captured image includes: Performing vehicle recognition on the captured image through a vehicle recognition model to obtain a target vehicle region of the target vehicle in the captured image; the vehicle recognition model is constructed based on a convolutional neural network; The target image is subjected to self-vehicle lane line recognition through a lane line recognition model to obtain the lane line position of the self-vehicle lane in the captured image; the lane line recognition model is constructed based on a convolutional neural network.

6. The method according to claim 1, characterized in that The fusing the target image area and the control response data of the vehicle to obtain the fusion features corresponding to the target vehicle includes: Extracting features from the target image region using a feature extraction model to obtain target image features; The target image features and the control response data of the vehicle are fused through weight parameters to obtain fused features corresponding to the target vehicle.

7. The method according to any one of claims 1 to 6, characterized in that: The regulated response data of the vehicle includes the acceleration, jerk and steering wheel angular velocity of the vehicle.

8. A wire pressing detection device, characterized in that: The device comprises: A determination module, used to determine a target image area where a target vehicle is located from the captured image of the vehicle; the target vehicle is a vehicle located around the vehicle; A fusion module is used to fuse the target image area and the control response data of the ego vehicle to obtain a fusion feature corresponding to the target vehicle; the control response data of the ego vehicle is used to indicate a change in the motion state of the ego vehicle; A detection module is used to perform lane crossing detection based on the fusion feature to obtain a lane crossing detection result corresponding to the target vehicle; the lane crossing detection result indicates whether the target vehicle crosses the lane line of the vehicle lane where the vehicle is located.

9. A vehicle, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the method according to any one of claims 1 to 7.

Images