Auxiliary system, system, method and storage medium for training image recognition model

Abstract

The present invention provides an auxiliary system for a vehicle, a system for training an image recognition model, a vehicle including the same and a corresponding method, a computer device and a computer-readable storage medium. The auxiliary system includes: a real-time image acquisition unit, configured to acquire one or more real-time images of queue-jumping vehicles near the current vehicle; a real-time image analysis unit, configured to analyze the real-time images using a pre-trained image recognition model to determine the risk level of the queue-jumping behavior of the queue-jumping vehicle; and a control unit, configured to send a reminder message and / or control information for controlling the adjustment of the driving state to the current vehicle and / or the queue-jumping vehicle according to the risk level. By utilizing the solution of the present invention, the danger level of the queue-jumping behavior of the queue-jumping vehicle can be determined, and the vehicle can be reminded or the driving state of the vehicle can be adjusted in response to the danger level, thereby improving the driving safety of the vehicle.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and more specifically, to an auxiliary system for a vehicle, a system for training an image recognition model, a corresponding method, a computer device, and a computer-readable storage medium. Background Art

[0002] When driving on the road, vehicles may encounter other vehicles changing lanes or cutting in. Since the existing radar system of vehicles has a detection blind spot, and the existing intelligent assistance systems of some vehicles fail to provide timely queue-cutting reminders and corresponding controls, current vehicles may cause traffic accidents due to inaccurate detection of other vehicles cutting in or untimely response.

[0003] Therefore, it is necessary to provide a technical solution that can more efficiently determine the danger level of other vehicles cutting in line. Summary of the invention

[0004] In order to solve the above technical problems, the present invention at least proposes an auxiliary system for a vehicle, which can determine the danger level of the queue-cutting behavior of the queue-cutting vehicle based on the acquired image of the queue-cutting vehicle, and remind the vehicle and / or adjust the vehicle's driving status in response to the determined danger level of the queue-cutting behavior, thereby improving the safety of vehicle driving.

[0005] According to a first aspect of the present invention, there is provided an auxiliary system for a vehicle, wherein the auxiliary system comprises:

[0006] A real-time image acquisition unit, configured to acquire one or more real-time images of queue-jumping vehicles near the current vehicle;

[0007] A real-time image analysis unit configured to analyze the real-time image using a pre-trained image recognition model to determine a risk level of the queue-jumping behavior of the queue-jumping vehicle;

[0008] The control unit is configured to send a reminder message and / or control information for adjusting the driving state to the current vehicle and / or the queue-jumping vehicle according to the risk level.

[0009] In one embodiment, the real-time image analysis unit is further configured to:

[0010] Inputting the acquired real-time image into the pre-trained image recognition model;

[0011] According to the acquired real-time image, searching the pre-trained image recognition model for a historical image that matches the real-time image;

[0012] If a historical image matching the real-time image is found, the risk level of the queue-jumping behavior of the queue-jumping vehicle is determined according to the risk level corresponding to the matching historical image; or,

[0013] If no historical image matching the real-time image is found, the acquired real-time image is analyzed to determine the driving data of the current vehicle and the queue-jumping vehicle, and the driving trajectories of the current vehicle and the queue-jumping vehicle in a future set time period are simulated based on the determined driving data of the current vehicle and the queue-jumping vehicle to predict whether the current vehicle and the queue-jumping vehicle will collide in the future set time period, and the risk level of the queue-jumping behavior of the queue-jumping vehicle is determined at least based on the prediction result.

[0014] In one embodiment, the control unit is further configured to:

[0015] When the risk level is lower than or equal to a set risk threshold, sending a reminder message indicating the risk level of the queue-jumping behavior to the current vehicle and / or the queue-jumping vehicle; and / or

[0016] When the risk level is higher than a set risk threshold, control information for controlling the current vehicle and / or the queue-jumping vehicle to adjust their driving states is sent to the current vehicle and / or the queue-jumping vehicle.

[0017] According to a second aspect of the present invention, there is provided an assistance method for a vehicle, wherein the assistance method comprises:

[0018] Acquire one or more real-time images of queue-jumping vehicles near the current vehicle;

[0019] Analyzing the real-time image using a pre-trained image recognition model to determine a risk level of the queue-jumping behavior of the queue-jumping vehicle;

[0020] According to the risk level, a reminder message and / or control message for controlling the current vehicle and / or the queue-jumping vehicle to adjust their driving status are sent to the current vehicle and / or the queue-jumping vehicle.

[0021] In one embodiment, the analyzing the real-time image using a pre-trained image recognition model to determine the risk level of the queue-jumping behavior of the queue-jumping vehicle further comprises:

[0022] Inputting the acquired real-time image into the pre-trained image recognition model;

[0023] According to the acquired real-time image, searching the pre-trained image recognition model for a historical image that matches the real-time image;

[0024] If a historical image matching the real-time image is found, the risk level of the queue-jumping behavior of the queue-jumping vehicle is determined according to the risk level corresponding to the matching historical image; or,

[0025] If no historical image matching the real-time image is found, the acquired real-time image is analyzed to determine the driving data of the current vehicle and the queue-jumping vehicle, and the driving trajectories of the current vehicle and the queue-jumping vehicle in a future set time period are simulated based on the determined driving data of the current vehicle and the queue-jumping vehicle to predict whether the current vehicle and the queue-jumping vehicle will collide in the future set time period, and the risk level of the queue-jumping behavior of the queue-jumping vehicle is determined at least based on the prediction result.

[0026] In one embodiment, sending a reminder message and / or sending information for controlling the current vehicle and / or the queue-jumping vehicle to adjust their driving states according to the risk level further includes:

[0027] When the risk level is lower than or equal to a set risk threshold, sending a reminder message indicating the risk level of the queue-jumping behavior to the current vehicle and / or the queue-jumping vehicle; and / or

[0028] When the risk level is higher than a set risk threshold, control information for controlling the current vehicle and / or the queue-jumping vehicle to adjust their driving states is sent to the current vehicle and / or the queue-jumping vehicle.

[0029] According to a third aspect of the present invention, a system for training an image recognition model is provided, comprising:

[0030] A historical image acquisition unit configured to acquire one or more historical images of one or more host vehicles and queue-jumping vehicles in their vicinity;

[0031] a historical image analysis unit configured to analyze the historical image to determine the driving data of the host vehicle and the queue-jumping vehicle and result data of whether the host vehicle and the queue-jumping vehicle collide, and to set a corresponding risk level for the historical image at least according to the result data;

[0032] The image recognition model training unit is configured to train and generate the image recognition model based at least on the determined historical images and their corresponding risk levels.

[0033] In one embodiment, the historical image analysis unit is further configured to:

[0034] If it is determined according to the result data that the host vehicle and the queue-jumping vehicle in the historical image have collided, then a collision position and a collision time of the collision are recorded, and a risk level of the historical image is set according to the collision position and the collision time; and / or

[0035] If it is determined based on the result data that the main vehicle and the queue-jumping vehicle in the historical image have not collided, the nearest position between the main vehicle and the queue-jumping vehicle and the time of reaching the nearest position are determined based on the historical image, and the risk level of the historical image is set based on the nearest position and the time of reaching the nearest position.

[0036] In one embodiment, the historical image analysis unit is further configured to:

[0037] If the historical image does not include an image that can be used to determine whether the main vehicle and the queue-jumping vehicle collide, the driving trajectories of the main vehicle and the queue-jumping vehicle in the historical image are simulated to predict whether there is a collision between the main vehicle and the queue-jumping vehicle in the historical image.

[0038] According to a fourth aspect of the present invention, a method for training an image recognition model is provided, comprising:

[0039] Acquire one or more historical images of one or more host vehicles and nearby queue-jumping vehicles;

[0040] Analyze the historical images to determine the driving data of the host vehicle and the queue-jumping vehicle and result data of whether the host vehicle and the queue-jumping vehicle collide, and set a corresponding risk level for the historical images at least according to the result data;

[0041] The image recognition model is trained and generated based at least on the determined historical images and their corresponding risk levels.

[0042] In one embodiment, the analyzing the historical images to determine the driving data of the host vehicle and the queue-jumping vehicle and the result data of whether the host vehicle and the queue-jumping vehicle collide, further comprises:

[0043] If it is determined according to the result data that the host vehicle and the queue-jumping vehicle in the historical image have collided, then a collision position and a collision time of the collision are recorded, and a risk level of the historical image is set according to the collision position and the collision time; and / or

[0044] If it is determined based on the result data that the main vehicle and the queue-jumping vehicle in the historical image have not collided, the nearest position between the main vehicle and the queue-jumping vehicle and the time of reaching the nearest position are determined based on the historical image, and the risk level of the historical image is set based on the nearest position and the time of reaching the nearest position.

[0045] In one embodiment, the analyzing the historical images to determine the driving data of the host vehicle and the queue-jumping vehicle and the result data of whether the host vehicle and the queue-jumping vehicle collide, further comprises:

[0046] If the historical image does not include an image that can be used to determine whether the main vehicle and the queue-jumping vehicle collide, the driving trajectories of the main vehicle and the queue-jumping vehicle in the historical image are simulated to predict whether there is a collision between the main vehicle and the queue-jumping vehicle in the historical image.

[0047] According to a fifth aspect of the present invention, a vehicle is provided, comprising the assistance system according to the present invention.

[0048] According to a sixth aspect of the present invention, there is provided a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the method of the present invention when executing the computer program.

[0049] According to a seventh aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method of the present invention when executed by a processor.

[0050] In some of the above-mentioned solutions provided by the present invention, the pre-trained image recognition model can be generated by training using the system or method for training image recognition models disclosed in the present invention, and it can be obtained by training based on real road traffic historical images, thereby ensuring the reliability of the image recognition model, and then the auxiliary system can use the image recognition model to quickly analyze the acquired real-time images to determine the danger level of queue-jumping behavior, greatly improving the accuracy and efficiency of risk judgment. Therefore, the above-mentioned solution provided by the present invention can help vehicles to respond to the determined danger level of queue-jumping behavior in a timely manner to remind vehicles or adjust the driving status of vehicles, thereby improving the safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Non-limiting and non-exhaustive embodiments of the present invention are described by way of example with reference to the following drawings, in which:

[0052] Figure 1 is a schematic diagram showing an example application scenario to which the present invention is applicable;

[0053] Figure 2 is a schematic diagram showing an auxiliary system for a vehicle according to an embodiment of the present invention;

[0054] Figure 3 is a flow chart showing an assist method for a vehicle according to one embodiment of the present invention;

[0055] Figure 4 is a schematic diagram showing a system for training an image recognition model according to one embodiment of the present invention;

[0056] Figure 5 is a flow chart illustrating a method for training an image recognition model according to one embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] Figure 1The example scenario shown includes the current vehicle and the queue-jumping vehicle that is about to jump the queue, and there is a potential collision risk between the two. The auxiliary system disclosed in this article can be partially or entirely set on the current vehicle or on an intelligent terminal device or remote server that communicates with the current vehicle, so that the current vehicle can use a pre-trained image recognition model to analyze the acquired real-time image of the queue-jumping vehicle to determine the risk level of the queue-jumping behavior of the queue-jumping vehicle. The "pre-trained image recognition model" can be obtained by training the system for training image recognition models disclosed in this article. In one embodiment, the current vehicle may be provided with a receiving device for receiving the pre-trained image recognition model, for example, the pre-trained image recognition model is received by an application (APP) installed on a vehicle-mounted mobile terminal or an intelligent terminal device of a vehicle user. The above-mentioned current vehicle and / or queue-jumping vehicle can be a manually driven vehicle or an autonomous driving vehicle. A plurality of sensors or positioning devices can be installed on the vehicle, such as a camera device, a laser radar, a millimeter wave radar, an ultrasonic sensor, a vehicle-to-everything communication (V2X) device, a highly automated driving (HAD, Highly Automated Driving) map, etc. These sensors can detect the environment around the vehicle, such as surrounding objects, obstacles, infrastructure, etc. For example, based on the sensors, the driving data of other vehicles near the current vehicle can be detected, and based on the driving data of other vehicles, it can be determined whether there is a queue-jumping vehicle in the other nearby vehicles that intends to cut in front of the current vehicle. The following driving data about the vehicle can be obtained through the sensors, positioning devices and / or by communicating with the online server: the vehicle's position, turn signal information, vehicle head direction, vehicle's lateral speed, longitudinal speed, acceleration, lane marking information, and / or the lateral distance between the current vehicle and the queue-jumping vehicle, etc.

[0059] Figure 2 FIG. 2 shows an auxiliary system 200 for a vehicle according to an embodiment of the present invention. Figure 2 As shown, the assistance system 200 includes a real-time image acquisition unit 210 , a real-time image analysis unit 220 and a control unit 230 .

[0060] The real-time image acquisition unit 210 can be configured to acquire one or more real-time images of the queue-jumping vehicle near the current vehicle. In one embodiment, the real-time image acquisition unit can acquire the real-time image through the following camera device, the camera device is, for example, a camera installed on the current vehicle, or a camera installed on other vehicles that can acquire images around the current vehicle and can communicate with the current vehicle, or a camera on an infrastructure that can communicate with the current vehicle (for example, a surveillance camera installed on the road). Preferably, the real-time image acquisition unit can acquire the real-time image through one or more cameras installed on the current vehicle. When the vehicle is equipped with multiple cameras, the multiple cameras can be set at the front, both sides, top and rear of the current vehicle. Real-time images may include but are not limited to pictures or videos. The "current vehicle" does not specifically refer to a specific vehicle, and any vehicle that may be queued on the road can be applied as the current vehicle in this article. The queue-jumping vehicle is a vehicle that actively queues (for example, merges or inserts) in front of the current vehicle. The vicinity of the current vehicle refers to within a preset distance range of the current vehicle, for example, within 3m or 5m, but is not limited to this, and can be set according to specific circumstances.

[0061] The real-time image analysis unit 220 may be configured to analyze the real-time image using a pre-trained image recognition model to determine a risk level of the queue-jumping behavior of the queue-jumping vehicle.

[0062] In one embodiment, the acquired real-time image may be input into the pre-trained image recognition model, and then, based on the acquired real-time image, the pre-trained image recognition model may be searched for a historical image that matches the real-time image.

[0063] In one embodiment, the image recognition model can extract feature images from the input real-time image, and then screen the historical images based on the extracted feature images to find matching historical images. In one embodiment, the image recognition model can be, for example, a neural network-based image recognition model, which can first extract feature images from the input real-time image using a neural network, and then map the feature images to the neural network for classification, and then match the corresponding historical images based on the classification results. The "match" means that the real-time image and the historical image have the same or similar relative position and relative speed. Specifically, for example, the relative position and relative speed of the current vehicle and the queue-jumping vehicle in the real-time image are the same or similar to the relative position and relative speed of the current vehicle and the queue-jumping vehicle in the historical image.

[0064] Depending on the situation, the real-time image analysis unit 220 may find a historical image that matches the real-time image, or may not find a historical image that matches the real-time image.

[0065] In one case, if a historical image matching the real-time image is found, the risk level of the queue-jumping behavior of the queue-jumping vehicle is determined according to the risk level corresponding to the matching historical image. Here, the risk level of the historical image can be divided in a variety of possible forms or ways; for example, the risk level can be divided into level I, level II and level III in the following form, wherein level I indicates that the possibility of the queue-jumping vehicle colliding with the current vehicle is small, for example, the collision probability is 10%; level II indicates that the possibility of the queue-jumping vehicle colliding with the current vehicle is uncertain, for example, the collision probability is 50%; level III indicates that the possibility of the queue-jumping vehicle colliding with the current vehicle is large, for example, the collision probability is 80%.

[0066] Or in another case, if no historical image matching the real-time image is found, the acquired real-time image is analyzed to determine the driving data of the current vehicle and the queue-jumping vehicle, and the driving trajectory of the current vehicle and the queue-jumping vehicle in the future set time period is simulated according to the determined driving data of the current vehicle and the queue-jumping vehicle, so as to predict whether the current vehicle and the queue-jumping vehicle will collide in the future set time period, and at least determine the risk level of the queue-jumping behavior of the queue-jumping vehicle according to the prediction result. Here, the real-time image analysis unit 220 can analyze the acquired real-time image with the help of the online server of the traffic control department, or its own configured processing module, or other applicable processing modules to obtain the driving data and simulate the driving trajectory. Wherein, the real-time image is preferably a video that can record the driving data of the current vehicle and / or the queue-jumping vehicle. The future set time period can be adjusted according to the situation. For example, when the speed of the queue-jumping vehicle is faster, the future set time period can be set shorter, otherwise it can be set longer. The prediction result can include collision and non-collision, and the risk level corresponding to the prediction result of collision is higher than the risk level corresponding to the prediction result of non-collision.

[0067] The control unit 230 may be configured to send a reminder message and / or a message for controlling the current vehicle and / or the queue-jumping vehicle to adjust their driving states according to the risk level.

[0068] In one embodiment, the control unit 230 may be further configured to: when the risk level is lower than or equal to a set risk threshold, send a reminder message indicating the risk level of the queue-jumping behavior to the current vehicle and / or the queue-jumping vehicle; and / or when the risk level is higher than the set risk threshold, send control information to the current vehicle and / or the queue-jumping vehicle to control the adjustment of its driving state. Here, the set risk threshold may be set according to the risk level or according to an empirical value. Optionally, both the risk level and the set risk threshold may also be expressed in the form of probability. For example, if the risk threshold is set to 45%, when the risk level is less than or equal to 45%, queue-jumping behavior may be allowed without changing the driving state of the current vehicle, and only a reminder message may be sent to the current vehicle; when the risk level is greater than 45%, control information is required to control the current vehicle to slow down or even stop driving or change the driving direction. The reminder information and the control information can be presented in a variety of possible ways. For vehicle users of non-autonomous driving vehicles, they can be presented in one of the following ways or any combination: voice broadcast, holographic projection, augmented reality display, vehicle display terminal display, portable smart device (smartphone, smart watch, smart bracelet) worn by vehicle users. For autonomous driving vehicles, the carrier signal carrying the reminder information can be sent to the vehicle by wired transmission or wireless transmission, wherein the wireless transmission can be achieved, for example, through a mobile network or Wi-Fi. The control information may include instructions for controlling the normal driving, deceleration, emergency braking or steering of the current vehicle.

[0069] Figure 3 FIG. 3 shows an assist method 300 for a vehicle according to an embodiment of the present invention. Figure 3 As shown, the auxiliary method 300 for a vehicle includes:

[0070] S310, acquiring one or more real-time images of queue-jumping vehicles near the current vehicle;

[0071] S320, analyzing the real-time image using a pre-trained image recognition model to determine a risk level of the queue-jumping behavior of the queue-jumping vehicle;

[0072] S330: Send reminder information and / or control information for adjusting the driving status to the current vehicle and / or the queue-jumping vehicle according to the risk level.

[0073] It should be understood that the auxiliary method 300 for a vehicle is performed by the auxiliary system 200 described above. The specific features described above with respect to the auxiliary system for a vehicle can also be similarly applied to the auxiliary method for a vehicle and similarly expanded. For the sake of simplicity, they are not described in detail.

[0074] Figure 4 A system 400 for training an image recognition model according to an embodiment of the present invention is shown. The system 400 for training an image recognition model includes a historical image acquisition unit 410, a historical image analysis unit 420, and an image recognition model training unit 430, and the units are communicatively coupled. The system 400 for training an image recognition model can be partially or entirely arranged on a vehicle, or preferably arranged on a server that can communicate with the vehicle.

[0075] Specifically, the historical image acquisition unit 410 may be configured to acquire one or more historical images regarding one or more host vehicles and queue-jumping vehicles in their vicinity.

[0076] In one embodiment, the historical image acquisition unit 410 can acquire the historical image in any one or more of the following possible ways: (1) the historical image acquisition unit 410 is configured to communicate with a camera on an available infrastructure, for example, via a mobile network, thereby acquiring the historical image with the help of the camera on the infrastructure; (2) the historical image acquisition unit 410 is configured to connect to a source capable of providing road image information and acquire the historical image from the source, for example, the source can be an online server of a traffic control department; wherein the connection between the historical image acquisition unit 410 and the source can be a wired connection or a wireless connection; (3) the historical image acquisition unit 410 is configured to communicate with other vehicles traveling on the road, for example, via a mobile network or Wi-Fi, thereby acquiring the historical image with the help of the other vehicles. The historical image may include a picture or a video, preferably a video.

[0077] The historical image analysis unit 420 can be configured to analyze the historical images to determine the driving data of the host vehicle and the queue-jumping vehicle respectively and result data of whether the host vehicle and the queue-jumping vehicle collide, and set a corresponding risk level for the historical images at least based on the result data.

[0078] In one embodiment, the historical image analysis unit 420 can also analyze the acquired real-time image with the help of the online server of the traffic control department, or its own configured processing module, or other applicable processing modules to obtain the driving data and the result data. Here, taking the historical image as a traffic monitoring video as an example, the acquired driving data may include complete dynamic parameters of the queue-jumping process, such as but not limited to the front direction, speed, acceleration, lane identification information of the main vehicle or the queue-jumping vehicle, and / or the lateral distance change between the main vehicle and the queue-jumping vehicle. Optionally, a corresponding risk level can be set for the historical image based on the driving data, for example, by determining whether the speed of the queue-jumping vehicle is greater than a preset speed threshold to set a corresponding risk level for the historical image. Specifically, if the speed of the queue-jumping vehicle is greater than the preset threshold, a higher risk level is set for the historical image; if the speed of the queue-jumping vehicle is not greater than the preset threshold, a lower risk level is set for the historical image.

[0079] The result data may include information representing a collision or no collision. In one case, if it is determined according to the result data that the host vehicle and the queue-jumping vehicle in the historical image have collided, the collision position and collision time of the collision may be recorded, and the risk level of the historical image may be set according to the collision position and the collision time. Here, the collision position and the collision time may both be determined based on an analysis of the historical image.

[0080] Taking the historical image as a traffic monitoring video as an example, the historical image analysis unit 420 can perform image segmentation and feature image extraction on the video screen to determine the position where the main vehicle and the queue-jumping vehicle were collided. The collision time can be determined based on the time information of the video, for example, calculating the difference between the time point when the queue-jumping vehicle collides with the main vehicle and the time point when the queue-jumping vehicle starts to cut in. The risk level can be set as follows: the closer the collision position is to the door, the higher the risk level, and vice versa; the shorter the collision time, the higher the risk level, and vice versa; or the greater the collision speed, the higher the risk level, and vice versa.

[0081] In another case, if it is determined according to the result data that the main vehicle and the queue-jumping vehicle in the historical image have not collided, the closest position between the main vehicle and the queue-jumping vehicle and the time of reaching the closest position are determined according to the historical image, and the risk level of the historical image is set according to the closest position and the time of reaching the closest position. Here, the historical image analysis unit 420 can obtain the driving data of the main vehicle and the queue-jumping vehicle by analyzing the historical image, and calculate the closest position and the time when the main vehicle and the queue-jumping vehicle arrive at the closest position respectively according to the driving data. The risk level can be set as follows: the closer the time when the main vehicle and the queue-jumping vehicle arrive at the closest position is, the higher the risk level is, and vice versa; the closer the closest position is to the current position of the vehicle, the higher the risk level is, and vice versa.

[0082] In one embodiment, the historical image analysis unit 420 is further configured to: if the historical image does not include an image that can be used to determine whether the main vehicle and the queue-jumping vehicle collide, then simulate the driving trajectories of the main vehicle and the queue-jumping vehicle in the historical image to predict whether there is a collision between the main vehicle and the queue-jumping vehicle in the historical image. Here, the historical image analysis unit 420 can analyze the historical image with the help of the online server of the traffic control department, or its own configured processing module, or other applicable processing modules to obtain the driving data and simulate the driving trajectory. The description of the driving data can refer to the above description. If the driving trajectories intersect, it can be considered that a collision will occur between the main vehicle and the queue-jumping vehicle, otherwise not. The image recognition model training unit 430 is configured to train and generate the image recognition model based at least on the determined historical images and their corresponding risk levels.

[0083] The related extensions of the image recognition model can be referred to in conjunction with Figure 2 The described embodiments are not described in detail here. As known to those skilled in the art, the image recognition model can also be understood as a model defined by a machine learning algorithm for image recognition that can perform the functions defined herein (e.g., a neural network image recognition model or other known types of machine learning models).

[0084] In an optional embodiment, the system 400 for training the image recognition model can be used in combination with the auxiliary system 200. For example, when the image recognition model is trained and applied to the auxiliary system 200 of the present invention, if the real-time image acquired by the auxiliary system includes an image that does not match the historical image in the image recognition model, the system 400 for training the image recognition model can be trained based on the real-time image to optimize the image recognition model.

[0085] Figure 5 A method 500 for training an image recognition model according to an embodiment of the present invention is shown. Figure 5 As shown, the method 500 for training an image recognition model includes:

[0086] S510, acquiring one or more historical images of one or more host vehicles and queue-jumping vehicles nearby;

[0087] S520, analyzing the historical image to determine the driving data of the host vehicle and the queue-jumping vehicle respectively and result data of whether the host vehicle and the queue-jumping vehicle collide, and setting a corresponding risk level for the historical image at least according to the result data;

[0088] S530: Train and generate the image recognition model based at least on the determined historical images and their corresponding risk levels.

[0089] It should be understood that the method 500 for training an image recognition model can be performed by the system 400 for training an image recognition model described above. The specific features described above in relation to the system for training an image recognition model can also be similarly applied to the method for training an image recognition model, and similar extensions are made. For the sake of simplicity, a detailed description thereof is not given.

[0090] It should be understood that the various units of the auxiliary system and the system for training the image recognition model of the present invention can be implemented in whole or in part by software, hardware, firmware or a combination thereof. Each of the units can be embedded in the processor of the computer device in the form of hardware or firmware or independent of the processor, or can be stored in the memory of the computer device in the form of software for the processor to call to perform the operation of each unit. Each of the units can be implemented as an independent component or module, or two or more units can be implemented as a single component or module.

[0091] It should be understood by those skilled in the art that Figure 2 A schematic diagram of the auxiliary system shown in FIG. Figure 4 The schematic diagram of the system for training the image recognition model is merely an exemplary block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device, processor or computer program embodying the solution of the present invention. A specific computer device, processor or computer program may include more or fewer components or modules than shown in the figure, or combine or split certain components or modules, or may have a different arrangement of components or modules.

[0092] In one embodiment, a computer device is provided, which includes a memory and a processor, wherein the memory stores computer instructions executable by the processor, and the computer instructions instruct the processor to perform each step of the auxiliary method of the present invention or the method for training an image recognition model when executed by the processor. The computer device can be a server, a vehicle-mounted terminal, or any other electronic device with necessary computing and / or processing capabilities in a broad sense. In one embodiment, the computer device may include a processor, a memory, a network interface, a communication interface, etc. connected via a system bus. The processor of the computer device can be used to provide necessary computing, processing and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. An operating system, a computer program, etc. may be stored in or on the non-volatile storage medium. The internal memory can provide an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface and the communication interface of the computer device can be used to connect and communicate with external devices through a network. When the computer program is executed by the processor, the steps of the auxiliary method of the present invention and the method for training an image recognition model are executed.

[0093] The present invention can be implemented as a computer-readable storage medium having a computer program stored thereon, which causes the steps of the method of the present invention to be performed when executed by a processor. In one embodiment, the computer program is distributed on a plurality of computer devices or processors coupled to a network so that the computer program is stored, accessed and executed in a distributed manner by one or more computer devices or processors. One or more method steps / operations can be performed by one or more computer devices or processors, and one or more other method steps / operations can be performed by one or more other computer devices or processors. One or more computer devices or processors can perform a single method step / operation, or perform two or more method steps / operations.

[0094] It will be appreciated by those skilled in the art that all or part of the steps of the auxiliary method of the present invention and the method for training an image recognition model may be performed by instructing related hardware such as a computer device or a processor through a computer program, and the computer program may be stored in a non-transitory computer-readable storage medium, and when the computer program is executed, the steps of the auxiliary method of the present invention or the method for training an image recognition model are performed. Depending on the circumstances, any reference to memory, storage, database or other media herein may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magnetic storage device, optical storage device, hard disk, solid state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0095] The various technical features described above can be combined arbitrarily. Although all possible combinations of these technical features are not described, any combination of these technical features should be considered to be covered by this specification as long as there is no contradiction in such combination.

[0096] Although the present invention has been described in conjunction with the embodiments, it will be appreciated by those skilled in the art that the above description and the accompanying drawings are exemplary and non-restrictive, and the present invention is not limited to the disclosed embodiments. Various modifications and variations are possible without departing from the spirit of the present invention.

Claims

1. An auxiliary system for a vehicle, characterized in that: The auxiliary system includes: A real-time image acquisition unit, configured to acquire one or more real-time images of queue-jumping vehicles near the current vehicle; A real-time image analysis unit configured to analyze the real-time image using a pre-trained image recognition model to determine a risk level of the queue-jumping behavior of the queue-jumping vehicle; a control unit configured to send a reminder message and / or control information for controlling the current vehicle and / or the queue-jumping vehicle to adjust their driving states according to the risk level, Wherein, the real-time image analysis unit is further configured to: Inputting the acquired real-time image into the pre-trained image recognition model; According to the acquired real-time image, searching the pre-trained image recognition model for a historical image that matches the real-time image; If a historical image matching the real-time image is found, the risk level of the queue-jumping behavior of the queue-jumping vehicle is determined according to the risk level corresponding to the matching historical image.

2. The assistance system according to claim 1, wherein: The real-time image analysis unit is further configured to: If no historical image matching the real-time image is found, the acquired real-time image is analyzed to determine the driving data of the current vehicle and the queue-jumping vehicle, and the driving trajectories of the current vehicle and the queue-jumping vehicle in a future set time period are simulated based on the determined driving data of the current vehicle and the queue-jumping vehicle to predict whether the current vehicle and the queue-jumping vehicle will collide in the future set time period, and the risk level of the queue-jumping behavior of the queue-jumping vehicle is determined at least based on the prediction result.

3. The assistance system according to claim 1 or 2, wherein: The control unit is further configured to: When the risk level is lower than or equal to a set risk threshold, a reminder message of the risk level of the queue-jumping behavior is sent to the current vehicle and / or the queue-jumping vehicle; and / or When the risk level is higher than a set risk threshold, control information for controlling the current vehicle and / or the queue-jumping vehicle to adjust their driving states is sent to the current vehicle and / or the queue-jumping vehicle.

4. A method for assisting a vehicle, characterized in that: The auxiliary method includes: Acquire one or more real-time images of queue-jumping vehicles near the current vehicle; Analyzing the real-time image using a pre-trained image recognition model to determine a risk level of the queue-jumping behavior of the queue-jumping vehicle; Sending a reminder message and / or a control message for adjusting the driving state to the current vehicle and / or the queue-jumping vehicle according to the risk level; The step of analyzing the real-time image using a pre-trained image recognition model to determine the risk level of the queue-jumping behavior of the queue-jumping vehicle further includes: Inputting the acquired real-time image into the pre-trained image recognition model; According to the acquired real-time image, searching the pre-trained image recognition model for a historical image that matches the real-time image; If a historical image matching the real-time image is found, the risk level of the queue-jumping behavior of the queue-jumping vehicle is determined according to the risk level corresponding to the matching historical image.

5. The assisting method according to claim 4, wherein: The analyzing the real-time image using a pre-trained image recognition model to determine the risk level of the queue-jumping behavior of the queue-jumping vehicle further includes: If no historical image matching the real-time image is found, the acquired real-time image is analyzed to determine the driving data of the current vehicle and the queue-jumping vehicle, and the driving trajectories of the current vehicle and the queue-jumping vehicle in a future set time period are simulated based on the determined driving data of the current vehicle and the queue-jumping vehicle to predict whether the current vehicle and the queue-jumping vehicle will collide in the future set time period, and the risk level of the queue-jumping behavior of the queue-jumping vehicle is determined at least based on the prediction result.

6. The assisting method according to claim 4 or 5, wherein: The sending of reminder information and / or information for controlling the adjustment of the driving state to the current vehicle and / or the queue-jumping vehicle according to the risk level further includes: When the risk level is lower than or equal to a set risk threshold, sending a reminder message indicating the risk level of the queue-jumping behavior to the current vehicle and / or the queue-jumping vehicle; and / or When the risk level is higher than a set risk threshold, control information for controlling the current vehicle and / or the queue-jumping vehicle to adjust their driving states is sent to the current vehicle and / or the queue-jumping vehicle.

7. A system for training an image recognition model, characterized in that: include: A historical image acquisition unit configured to acquire one or more historical images of one or more host vehicles and queue-jumping vehicles in their vicinity; a historical image analysis unit configured to analyze the historical image to determine the driving data of the host vehicle and the queue-jumping vehicle and result data of whether the host vehicle and the queue-jumping vehicle collide, and to set a corresponding risk level for the historical image at least according to the result data; an image recognition model training unit, configured to train and generate the image recognition model based at least on the determined historical images and their corresponding risk levels, The image recognition model can extract feature images from the real-time image, and then screen the historical images based on the extracted feature images to find matching historical images that match the real-time image, wherein the real-time image is a real-time image of a queue-jumping vehicle near the current vehicle. If a matching historical image matching the real-time image is found, the risk level of the queue-jumping behavior of the queue-jumping vehicle near the current vehicle is determined according to the risk level corresponding to the matching historical image.

8. The system for training an image recognition model according to claim 7, wherein: The historical image analysis unit is further configured to: If it is determined according to the result data that the host vehicle and the queue-jumping vehicle in the historical image have collided, the collision position and collision time of the collision are recorded, and the risk level of the historical image is set according to the collision position and the collision time; and / or If it is determined based on the result data that the main vehicle and the queue-jumping vehicle in the historical image have not collided, the nearest position between the main vehicle and the queue-jumping vehicle and the time of reaching the nearest position are determined based on the historical image, and the risk level of the historical image is set based on the nearest position and the time of reaching the nearest position.

9. The system for training an image recognition model according to claim 7 or 8, wherein: The historical image analysis unit is further configured to: If the historical image does not include an image that can be used to determine whether the main vehicle and the queue-jumping vehicle collide, the driving trajectories of the main vehicle and the queue-jumping vehicle in the historical image are simulated to predict whether there is a collision between the main vehicle and the queue-jumping vehicle in the historical image.

10. A method for training an image recognition model, characterized in that: include: Acquire one or more historical images of one or more host vehicles and nearby queue-jumping vehicles; Analyze the historical images to determine the driving data of the host vehicle and the queue-jumping vehicle and result data of whether the host vehicle and the queue-jumping vehicle collide, and set a corresponding risk level for the historical images at least according to the result data; The image recognition model is trained and generated based at least on the determined historical images and their corresponding risk levels, The image recognition model can extract feature images from the real-time image, and then screen the historical images based on the extracted feature images to find matching historical images that match the real-time image, wherein the real-time image is a real-time image of a queue-jumping vehicle near the current vehicle. If a matching historical image matching the real-time image is found, the risk level of the queue-jumping behavior of the queue-jumping vehicle near the current vehicle is determined according to the risk level corresponding to the matching historical image.

11. The method for training an image recognition model according to claim 10, wherein: The analyzing the historical images to determine the driving data of the host vehicle and the queue-jumping vehicle and the result data of whether the host vehicle and the queue-jumping vehicle collided further includes: If it is determined according to the result data that the host vehicle and the queue-jumping vehicle in the historical image have collided, then a collision position and a collision time of the collision are recorded, and a risk level of the historical image is set according to the collision position and the collision time; and / or If it is determined based on the result data that the main vehicle and the queue-jumping vehicle in the historical image have not collided, the nearest position between the main vehicle and the queue-jumping vehicle and the time of reaching the nearest position are determined based on the historical image, and the risk level of the historical image is set based on the nearest position and the time of reaching the nearest position.

12. The method for training an image recognition model according to claim 10 or 11, wherein: The analyzing the historical images to determine the driving data of the host vehicle and the queue-jumping vehicle and the result data of whether the host vehicle and the queue-jumping vehicle collided further includes: If the historical image does not include an image that can be used to determine whether the main vehicle and the queue-jumping vehicle collide, the driving trajectories of the main vehicle and the queue-jumping vehicle in the historical image are simulated to predict whether there is a collision between the main vehicle and the queue-jumping vehicle in the historical image.

13. A vehicle, characterized in that: Comprising an auxiliary system according to any one of claims 1-3.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method of any one of claims 4 to 6 or any one of claims 10 to 12 is implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program implements the method of any one of claims 4 to 6 or any one of claims 10 to 12 when executed by a processor.

Images