Tire protection method, device and equipment based on machine vision and program product

Abstract

The invention discloses a tire protection method, device and equipment based on machine vision and a program product. The method comprises the following steps: acquiring road image information and radar detection information of a road in front of a current vehicle; inputting the road image information and the radar detection information into a pre-trained obstacle recognition model to obtain an obstacle type, an obstacle size and an obstacle orientation; predicting a target tire which will be in contact with the obstacle according to the azimuth of the obstacle; determining a tire protection operation on the target tire according to the obstacle type and the obstacle size, and executing the tire protection operation through the current vehicle; wherein an electromagnetic adsorption device and a powerful dust collection device are arranged on a chassis around tires of the current vehicle, and an active suspension system of the current vehicle has a tire lifting function. The tire can be protected when the vehicle runs, the tire is prevented from being damaged, so that the safety of vehicle running is improved, and the tire protection device can be applied to the technical field of vehicle monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle monitoring, and in particular to a tire protection method, device, equipment and program product based on machine vision. BACKGROUND

[0002] During driving, the tires of a vehicle can be damaged to varying degrees due to various conditions of the road surface. For example, discarded foreign objects on the road surface can puncture or scratch the tires, and road surface depressions / protuberances can cause the vehicle to jolt, which can lead to tire bulging and suspension damage, affecting the safety of vehicle driving.

[0003] The above problems need to be solved. SUMMARY

[0004] The present application aims to at least partially solve one of the technical problems existing in the prior art.

[0005] To this end, one purpose of an embodiment of the present application is to provide a tire protection method based on machine vision, which can protect the tires during vehicle driving and avoid tire damage, thereby improving the safety of vehicle driving.

[0006] Another purpose of an embodiment of the present application is to provide a tire protection device based on machine vision.

[0007] In order to achieve the above technical purposes, the technical solutions adopted by the embodiments of the present application include: On the one hand, an embodiment of the present application provides a tire protection method based on machine vision, comprising the following steps: obtaining road image information and radar detection information of a road ahead of a current vehicle; inputting the road image information and the radar detection information into a pre-trained obstacle recognition model to obtain an obstacle type, an obstacle size and an obstacle orientation; predicting a target tire that will contact the obstacle according to the obstacle orientation; determining a tire protection operation for the target tire according to the obstacle type and the obstacle size, and executing the tire protection operation by the current vehicle; wherein an electromagnetic adsorption device and a powerful suction device are arranged around the tires of the current vehicle, the active suspension system of the current vehicle has a tire lifting function, and the tire protection operation includes at least one of starting the electromagnetic adsorption device around the tires, starting the powerful suction device around the tires, starting the tire lifting function, and reducing speed and detouring.

[0008] Further, in an embodiment of the present application, the obstacle recognition model is trained by the following steps: obtain a road image sample and a radar detection sample of a road in front of a test vehicle; perform time alignment and time series processing on the road image sample and the radar detection sample to obtain road image time series data and radar point cloud time series data; construct a training sample according to the road image time series data and the radar point cloud time series data, and determine a corresponding obstacle label through manual labeling to obtain a training data set; input the training data set into a pre-constructed CNN-LSTM hybrid neural network for training to obtain the trained obstacle recognition model; The obstacle label includes an obstacle type label, an obstacle size label, and an obstacle orientation label.

[0009] Further, in an embodiment of the present application, the CNN-LSTM hybrid neural network includes an image convolution branch, a point cloud convolution branch, a feature fusion layer, an LSTM layer, and an output layer. The input of the training data set into the pre-constructed CNN-LSTM hybrid neural network for training to obtain the trained obstacle recognition model specifically includes: perform convolution processing on the road image time series data through the image convolution branch to obtain image spatial features; perform convolution processing on the radar point cloud time series data through the point cloud convolution branch to obtain point cloud spatial features; perform feature fusion on the image spatial features and the point cloud spatial features based on an attention mechanism through the feature fusion layer to obtain global fusion features; generate spatio-temporal fusion features according to the global fusion features of multiple consecutive time steps; perform hidden state calculation on the spatio-temporal fusion features through the LSTM layer to obtain a hidden state vector; map the hidden state vector to the obstacle recognition result through the output layer; determine a loss value according to the obstacle recognition result and the obstacle label; update the CNN-LSTM hybrid neural network according to the loss value to obtain the trained obstacle recognition model.

[0010] Further, in an embodiment of the present application, the prediction of the target tire that will contact the obstacle according to the obstacle orientation specifically includes: obtain a current vehicle speed, a current acceleration, and a current steering angle of the current vehicle; predict a vehicle trajectory of the current vehicle according to the current vehicle speed, the current acceleration, and the current steering angle; determining a tire travel trajectory of each tire according to the tire distribution position of the current vehicle and the vehicle travel trajectory; predicting the target tire that will contact the obstacle according to the obstacle orientation and the tire travel trajectory.

[0011] Further, in an embodiment of the present application, the tire protection operation determined according to the obstacle type and the obstacle size specifically includes: when the obstacle type is a metal object and the obstacle size is less than or equal to a preset first size threshold, determining the tire protection operation as starting an electromagnetic adsorption device and a powerful suction device around the tire; when the obstacle type is a non-metal object and the obstacle size is less than or equal to the first size threshold, determining the tire protection operation as starting a powerful suction device around the tire; when the obstacle type is a road surface depression or a road surface protrusion and the obstacle size is less than or equal to a preset second size threshold, determining the tire protection operation as starting a tire lifting function; when the obstacle type is a metal object or a non-metal object and the obstacle size is greater than the first size threshold, determining the tire protection operation as decelerating and bypassing; when the obstacle type is a road surface depression or a road surface protrusion and the obstacle size is greater than the second size threshold, determining the tire protection operation as decelerating and bypassing.

[0012] Further, in an embodiment of the present application, the tire protection method further includes the following steps: when the tire protection operation is starting a tire lifting function or decelerating and bypassing, generating obstacle warning information according to the obstacle type, the obstacle size, and the obstacle orientation, and sharing the obstacle warning information to a navigation platform.

[0013] Further, in an embodiment of the present application, the tire protection method further includes the following steps: when the tire protection operation is starting a tire lifting function or decelerating and bypassing, generating obstacle reporting information according to the obstacle type, the obstacle size, the obstacle orientation, and the road image information, and sending the obstacle reporting information to a road administration system.

[0014] On the other hand, an embodiment of the present application provides a tire protection device based on machine vision, comprising: an information acquisition module, configured to acquire road image information and radar detection information of a road ahead of a current vehicle; an obstacle recognition module, configured to input the road image information and the radar detection information into a pre-trained obstacle recognition model to obtain an obstacle type, an obstacle size, and an obstacle orientation; a target tire determination module, configured to predict a target tire that will contact the obstacle according to the obstacle orientation; a protection operation execution module, configured to determine a tire protection operation for the target tire according to the obstacle type and the obstacle size, and execute the tire protection operation by the current vehicle; wherein a chassis around a tire of the current vehicle is provided with an electromagnetic adsorption device and a powerful dust suction device, an active suspension system of the current vehicle is provided with a tire lifting function, and the tire protection operation includes at least one of starting the electromagnetic adsorption device around the tire, starting the powerful dust suction device around the tire, starting the tire lifting function, and decelerating and bypassing.

[0015] In another aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor; at least one memory configured to store at least one program; when the at least one program is executed by the at least one processor, the at least one processor is caused to implement the above-mentioned machine vision-based tire protection method.

[0016] In another aspect, an embodiment of the present application further provides a computer readable storage medium, wherein a computer program executable by a processor is stored, and the computer program, when executed by the processor, implements the above-mentioned machine vision-based tire protection method.

[0017] In another aspect, an embodiment of the present application further provides a computer program product, comprising a computer program, and the computer program, when executed by a processor, implements the above-mentioned machine vision-based tire protection method.

[0018] The advantages and beneficial effects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be understood through the practice of the present application: The embodiment of the present application obtains road image information and radar detection information of a road in front of a current vehicle, inputs the road image information and the radar detection information into a pre-trained obstacle recognition model, obtains an obstacle type, an obstacle size and an obstacle direction, predicts a target tire that will contact the obstacle according to the obstacle direction, determines a tire protection operation on the target tire according to the obstacle type and the obstacle size, and executes the tire protection operation by the current vehicle, wherein a chassis around a tire of the current vehicle is provided with an electromagnetic adsorption device and a powerful dust suction device, an active suspension system of the current vehicle is provided with a tire lifting function, and the tire protection operation includes at least one of starting the electromagnetic adsorption device around the tire, starting the powerful dust suction device around the tire, starting the tire lifting function, and decelerating and bypassing. The embodiment of the present application is provided with the electromagnetic adsorption device and the powerful dust suction device around the chassis of the tire of the vehicle, and the active suspension system of the vehicle is provided with the tire lifting function, the obstacle type, the obstacle size and the obstacle direction are recognized according to the road image information and the radar detection information of the road in front of the vehicle when the vehicle is running, the target tire that will contact the obstacle is predicted based on the obstacle direction, the most appropriate tire protection operation is selected according to the obstacle type and the obstacle size, the tire can be protected when the vehicle is running, the tire is prevented from being damaged, and the safety of the vehicle running is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following introduces the drawings needed to be used in the embodiments of the present application as follows. It should be understood that the drawings introduced in the following are only for facilitating the clear description of the technical solutions in some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative labor on the basis of these drawings.

[0020] Figure 1 A step flow chart of a tire protection method based on machine vision provided by the embodiment of the present application; Figure 2 A structural block diagram of a tire protection device based on machine vision provided by the embodiment of the present application; Figure 3 A structural block diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the present application. They are only examples of apparatuses and methods consistent with some aspects of embodiments of the present application as detailed in the appended claims.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to limit the present application.

[0023] The machine vision-based tire protection method provided by the embodiments of the present application can be applied to a terminal, can be applied to a server, and can also be software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, and the like, but is not limited thereto. The server end can be configured as a stand-alone physical server, can be configured as a server cluster or a distributed system formed by multiple physical servers, can be configured as a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms, and the server can also be a node server in a blockchain network. The software can be an application that implements the machine vision-based tire protection method, and the like, but is not limited to the above forms.

[0024] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0025] It should be noted that in various specific embodiments of the present application, when relevant processing needs to be performed on data related to the identity or characteristics of the user, such as user information, user behavior data, user history data, and user location information, the user's permission or consent is obtained first, and the collection, use, and processing of such data comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the user's separate permission or separate consent is obtained through a pop-up window or a jump to a confirmation page, and after obtaining the user's separate permission or separate consent, the necessary user-related data for the normal operation of the embodiments of the present application is obtained.

[0026] Reference Figure 1 The embodiments of the present application provide a tire protection method based on machine vision, which specifically comprises the following steps: S101, obtaining road image information and radar detection information of a road in front of a current vehicle; S102, inputting the road image information and the radar detection information into a pre-trained obstacle recognition model to obtain an obstacle type, an obstacle size, and an obstacle orientation; S103, predicting a target tire that will contact the obstacle according to the obstacle orientation; S104, determining a tire protection operation for the target tire according to the obstacle type and the obstacle size, and performing the tire protection operation by the current vehicle; Wherein, the chassis around the tire of the current vehicle is provided with an electromagnetic adsorption device and a powerful dust suction device, the active suspension system of the current vehicle has a tire lifting function, and the tire protection operation includes at least one of starting the electromagnetic adsorption device around the tire, starting the powerful dust suction device around the tire, starting the tire lifting function, and decelerating and bypassing.

[0027] The embodiments of the present application are provided with an electromagnetic adsorption device and a powerful dust suction device around the chassis of the tire of the vehicle, and the active suspension system of the vehicle has a tire lifting function. The obstacle type, the obstacle size, and the obstacle orientation are recognized according to the road image information and the radar detection information of the road in front of the vehicle when the vehicle is running, the target tire that will contact the obstacle is predicted based on the obstacle orientation, and the most appropriate tire protection operation is selected according to the obstacle type and the obstacle size. The tire can be protected when the vehicle is running, and the tire can be protected from damage, thereby improving the safety of the vehicle in running.

[0028] Further as an optional embodiment, the obstacle recognition model is trained by the following steps: S201, obtaining road image samples and radar detection samples of a road in front of a test vehicle; S202, time alignment and time series processing are performed on the road image sample and the radar detection sample to obtain road image time series data and radar point cloud time series data; S203, training samples are constructed according to the road image time series data and the radar point cloud time series data, and corresponding obstacle labels are determined by manual labeling to obtain a training data set; S204, the training data set is input into a pre-constructed CNN-LSTM hybrid neural network for training to obtain a trained obstacle recognition model; The obstacle label includes an obstacle type label, an obstacle size label, and an obstacle orientation label.

[0029] Specifically, a high-definition detection camera with infrared function is installed at the front end of the vehicle head, a millimeter wave radar and a laser radar are installed at the lower end of the vehicle head, the vehicle head camera captures the road image in front of the vehicle, and the millimeter wave radar and the laser radar detect the road obstacles in front to obtain point cloud data.

[0030] For the road image sample and the radar detection sample of the test vehicle, time alignment and time series processing are performed, that is, road image time series data and radar point cloud time series data are obtained, which constitute a training sample. At the same time, the corresponding obstacle label is determined by manual labeling, including the obstacle type label, the obstacle size label, and the obstacle orientation label. The training data set is input into the pre-constructed CNN-LSTM hybrid neural network for training, and the trained obstacle recognition model is obtained.

[0031] Further as an optional implementation, the CNN-LSTM hybrid neural network includes an image convolution branch, a point cloud convolution branch, a feature fusion layer, an LSTM layer, and an output layer. The training data set is input into the pre-constructed CNN-LSTM hybrid neural network for training to obtain the trained obstacle recognition model, which specifically includes: S2041, the image convolution branch is used to perform convolution processing on the road image time series data to obtain image spatial features; S2042, the point cloud convolution branch is used to perform convolution processing on the radar point cloud time series data to obtain point cloud spatial features; S2043, the feature fusion layer is used to perform feature fusion on the image spatial features and the point cloud spatial features based on an attention mechanism to obtain global fusion features; S2044, spatio-temporal fusion features are generated according to the global fusion features of multiple continuous time steps; S2045, the LSTM layer is used to perform hidden state calculation on the spatio-temporal fusion features to obtain a hidden state vector; S2046, the output layer is used to map the hidden state vector to an obstacle recognition result; S2047, determine a loss value according to the obstacle recognition result and the obstacle label; S2048, update the CNN-LSTM hybrid neural network according to the loss value to obtain the trained obstacle recognition model.

[0032] Specifically, the image spatial features are obtained by performing convolution processing on the road image time series data through the image convolution branch, the point cloud spatial features are obtained by performing convolution processing on the radar point cloud time series data through the point cloud convolution branch, the image spatial features and the point cloud spatial features are fused based on the attention mechanism through the feature fusion layer to obtain the global fusion features; the spatio-temporal fusion features are generated according to the global fusion features of multiple continuous time steps, the hidden state vector is obtained by performing hidden state calculation on the spatio-temporal fusion features through the LSTM layer, and the obstacle recognition result is obtained by mapping the hidden state vector through the output layer; the loss value is determined according to the obstacle recognition result and the obstacle label, the CNN-LSTM hybrid neural network is updated according to the loss value, and one round of iterative training is completed; when the number of iterations reaches a preset threshold value or the loss value is lower than a preset threshold value, the training is stopped, and the trained obstacle recognition model is obtained.

[0033] Further, as an optional implementation, the target tire that will contact the obstacle is predicted according to the obstacle orientation, and the target tire is determined according to the tire distribution position of the current vehicle and the vehicle trajectory, and the target tire is determined according to the tire distribution position of the current vehicle and the vehicle trajectory. S1031, obtain the current speed, the current acceleration and the current steering angle of the current vehicle; S1032, predict the vehicle trajectory of the current vehicle according to the current speed, the current acceleration and the current steering angle; S1033, determine the tire trajectory of each tire according to the tire distribution position of the current vehicle and the vehicle trajectory; S1034, predict the target tire that will contact the obstacle according to the obstacle orientation and the tire trajectory.

[0034] Specifically, the vehicle trajectory of the current vehicle is predicted based on the current speed, the current acceleration and the current steering angle, and then the tire trajectory of each tire is determined according to the tire distribution position of the vehicle, and the target tire is determined according to the tire trajectory and the obstacle orientation.

[0035] Further, as an optional implementation, the tire protection operation on the target tire is determined according to the obstacle type and the obstacle size, and the tire protection operation on the target tire is determined according to the obstacle type and the obstacle size. S1041, when the obstacle type is a metal object and the obstacle size is less than or equal to a preset first size threshold, the tire protection operation is determined as starting the electromagnetic adsorption device and the powerful dust collection device around the tire; S1042, when the obstacle type is a non-metallic object and the obstacle size is less than or equal to the first size threshold, determining the tire protection operation as starting a powerful suction device around the tire; S1043, when the obstacle type is a road depression or a road protrusion and the obstacle size is less than or equal to a preset second size threshold, determining the tire protection operation as starting a tire lifting function; S1044, when the obstacle type is a metallic object or a non-metallic object and the obstacle size is greater than the first size threshold, determining the tire protection operation as decelerating and bypassing; S1045, when the obstacle type is a road depression or a road protrusion and the obstacle size is greater than the second size threshold, determining the tire protection operation as decelerating and bypassing.

[0036] Specifically, when a metallic foreign object (such as a nail or a bolt) with a size less than or equal to the first size threshold is identified, an electromagnetic adsorption device (magnetic strength ≥ 15000 Gauss) and a powerful suction device (wind pressure 8000 Pa) around the target tire are started synchronously to form double protection; when a non-metallic foreign object (such as glass fragments or ceramic debris) with a size less than or equal to the first size threshold is identified, a three-stage suction mode of the powerful suction device is started to realize solid-gas separation through cyclone separation technology; when a road depression or a road protrusion with a size less than or equal to the second size threshold is identified, the vehicle will lift the target tire through the active suspension system, so that the target tire does not fall into the depression area or does not contact the protrusion area; when the identified metallic or non-metallic foreign object is large in size, or the identified road depression or protrusion is large in size, the ADAS system immediately triggers AEB automatic emergency braking (braking distance < 1.5 m / 60 km / h), and simultaneously activates steering avoidance assistance to generate a safe bypassing trajectory.

[0037] Further as an optional implementation, the tire protection method further comprises the following steps: S105, when the tire protection operation is to start the tire lifting function or decelerate and bypass, generating obstacle warning information according to the obstacle type, the obstacle size, and the obstacle direction, and sharing the obstacle warning information to a navigation platform.

[0038] Further as an optional implementation, the tire protection method further comprises the following steps: S106, when the tire protection operation is to start the tire lifting function or decelerate and bypass, generating obstacle reporting information according to the obstacle type, the obstacle size, the obstacle direction, and road image information, and sending the obstacle reporting information to a road administration system.

[0039] Specifically, when the tire protection operation is to start the tire lifting function or decelerate and detour, although the vehicle can avoid contacting the obstacle, the obstacle can still affect other vehicles on the road, so the obstacle warning information can be generated and shared to the navigation platform to remind other vehicles to avoid, and the obstacle reporting information can be sent to the road management system to facilitate the road management personnel to clean the road obstacle in time.

[0040] The above describes the method steps of the embodiment of the present application. It can be recognized that the electromagnetic adsorption device and the powerful dust collection device are arranged on the chassis around the tire of the vehicle, and the active suspension system of the vehicle is provided with the tire lifting function. The type, size and direction of the obstacle are identified according to the road image information and the radar detection information of the front road when the vehicle is driving. The target tire that will contact the obstacle is predicted based on the direction of the obstacle. The most appropriate tire protection operation is selected according to the type and size of the obstacle. The tire can be protected when the vehicle is driving, and the tire damage can be avoided, thereby improving the safety of the vehicle driving.

[0041] Reference Figure 2 The embodiment of the present application provides a tire protection device based on machine vision, which comprises: An information acquisition module is configured to acquire road image information and radar detection information of a road in front of a current vehicle. An obstacle identification module is configured to input the road image information and the radar detection information into a pre-trained obstacle identification model to obtain the type, size and direction of the obstacle. A target tire determination module is configured to predict a target tire that will contact the obstacle according to the direction of the obstacle. A protection operation execution module is configured to determine a tire protection operation for the target tire according to the type and size of the obstacle, and execute the tire protection operation through the current vehicle. The chassis around the tire of the current vehicle is provided with the electromagnetic adsorption device and the powerful dust collection device, the active suspension system of the current vehicle is provided with the tire lifting function, and the tire protection operation comprises at least one of starting the electromagnetic adsorption device around the tire, starting the powerful dust collection device around the tire, starting the tire lifting function and decelerating and detouring.

[0042] It can be understood that the contents in the above method embodiment are applicable to the device embodiment, the device embodiment specifically realizes the same functions as the above method embodiment, and achieves the same beneficial effects as the above method embodiment.

[0043] Reference Figure 3 The embodiment of the present application provides an electronic device, which comprises: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the machine vision-based tire protection method.

[0044] It can be understood that the contents in the above method embodiments are applicable to the present device embodiments, the present device embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0045] The present application embodiment further provides a computer readable storage medium, which stores a computer program executable by a processor, and the computer program implements the machine vision-based tire protection method when executed by the processor.

[0046] The computer readable storage medium of the present application embodiment can execute the machine vision-based tire protection method provided by the method embodiments of the present application, and can execute any combination of the method embodiments to implement the steps, and has the corresponding functions and beneficial effects of the method.

[0047] The present application embodiment further provides a computer program product, which includes a computer program, and the computer program implements the machine vision-based tire protection method when executed by a processor.

[0048] It can be understood that the contents in the above method embodiments are applicable to the present program product embodiments, the present program product embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0049] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0050] The embodiments described in the present application embodiments are used to more clearly illustrate the technical solutions of the present application embodiments, and do not constitute a limitation on the technical solutions provided by the present application embodiments. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the present application embodiments are also applicable to similar technical problems.

[0051] The terms "first", "second", "third", "fourth" etc. (if any) in the description and drawings of the application are used for distinguishing between similar objects, not necessarily described in the above described order, and are used solely to distinguish between two consecutive reference numerals used three or more times in any combination. It should be understood that the use of these terms herein does not imply a serial or chronological order among the items being referred to, but rather the terms "first", "second" and "third" are used to distinguish one object from another in a given set of objects. It should be further understood that the use of the terms first, second, third, etc., are likely used to indicate different aspects of an embodiment of the present application, but the terms first, second, third, etc. are not necessarily used chronologically to denote initial, secondary or tertiary discussion but are used to distinguish one aspect of an embodiment of the present application from another aspect of an embodiment of the present application. Similarly, it should be understood that the use of the terms "a", "an", "the" and "said" are intended to include the plural as well as the singular, unless the context clearly indicates otherwise. It will be further understood that the terms "comprise" and "comprising", when used in this specification, specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. It should be understood that the terms "comprise", "comprising", "comprises" and "comprising" when used in this specification, specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0052] In some alternative embodiments, the functions / operations mentioned in the block diagrams can not occur in the order mentioned in the operational illustrations. For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending on the functionality / operations involved. Also, the embodiments presented and described in the flowcharts are only examples. Alternative embodiments can be conceived and carried out by a person of ordinary skill in the art without departing from the scope of the present application. Accordingly, the disclosed methods are not limited to the operations and logical flows presented in the flowcharts. Alternative embodiments are possible, wherein the order of various operations is changed and wherein sub operations described as part of a larger operation are independently executed.

[0053] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the functions and / or features described above can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be appreciated that detailed discussion of the actual implementation of each module is not necessary to an understanding of the present application. Rather, the actual implementation of the modules will be apparent to those skilled in the art in view of the property, function and internal relationships of the various functional modules disclosed herein. Accordingly, the present application is not limited to the embodiments illustrated in the figures. Rather, alternative embodiments are possible, wherein the order of various operations is changed and wherein sub operations described as part of a larger operation are independently executed. It is also to be understood that the disclosed specific concepts are merely illustrative and not intended to limit the scope of the present application, which is defined by the appended claims and their equivalents.

[0054] If the above functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or parts of the technical solutions that essentially contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described above in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0055] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, which can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus or device, such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute the instructions, or in conjunction with these instruction execution systems, apparatus or devices. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by or in connection with an instruction execution system, apparatus or device, or in conjunction with these instruction execution systems, apparatus or devices.

[0056] More specific examples (non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CD ROM). In addition, the computer readable medium can even be paper or other suitable medium on which the above programs are printed, because the programs can be obtained electronically, for example, by optical scanning of the paper or other medium, followed by editing, interpreting or otherwise processing, if necessary, in other suitable ways, to be electronically obtained and then stored in the computer memory.

[0057] It should be understood that various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or their combinations can be used: a discrete logic circuit having logic gates for implementing logic functions on data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), or the like.

[0058] In the above description of the present specification, the description of the terms "one embodiment / one example", "another embodiment / another example", or "certain embodiments / certain examples" or the like means that the specific features, structures, materials or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0059] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made thereto without departing from the principles and spirit of the present application, the scope of which is defined by the claims and their equivalents.

[0060] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above-described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope defined by the claims of the present application.

Claims

1. A machine vision-based tire protection method, characterized by, The method comprises the following steps: obtaining road image information and radar detection information of a road ahead of a current vehicle; inputting the road image information and the radar detection information into a pre-trained obstacle recognition model to obtain an obstacle type, an obstacle size, and an obstacle orientation; predicting a target tire that will contact the obstacle according to the obstacle orientation; determining a tire protection operation for the target tire according to the obstacle type and the obstacle size, and performing the tire protection operation by the current vehicle; wherein a chassis around a tire of the current vehicle is provided with an electromagnetic adsorption device and a powerful dust suction device, an active suspension system of the current vehicle is provided with a tire lifting function, and the tire protection operation comprises at least one of starting the electromagnetic adsorption device around the tire, starting the powerful dust suction device around the tire, starting the tire lifting function, and decelerating and bypassing.

2. The method of claim 1, wherein the tire protection method is based on machine vision. The obstacle recognition model is obtained by the following steps: obtaining road image samples and radar detection samples of a road ahead of a test vehicle; performing time alignment and time sequence processing on the road image samples and the radar detection samples to obtain road image time sequence data and radar point cloud time sequence data; constructing training samples according to the road image time sequence data and the radar point cloud time sequence data, and determining corresponding obstacle labels by manual labeling to obtain a training data set; inputting the training data set into a pre-constructed CNN-LSTM hybrid neural network for training to obtain the trained obstacle recognition model; wherein the obstacle labels comprise an obstacle type label, an obstacle size label, and an obstacle orientation label.

3. The method of claim 2, wherein the tire protection method is based on machine vision. The CNN-LSTM hybrid neural network comprises an image convolution branch, a point cloud convolution branch, a feature fusion layer, an LSTM layer, and an output layer, and the training of the training data set into the pre-constructed CNN-LSTM hybrid neural network to obtain the trained obstacle recognition model specifically comprises: performing convolution processing on the road image time sequence data through the image convolution branch to obtain image spatial features; performing convolution processing on the radar point cloud time sequence data through the point cloud convolution branch to obtain point cloud spatial features; performing feature fusion on the image spatial features and the point cloud spatial features based on an attention mechanism through the feature fusion layer to obtain global fusion features; generating spatio-temporal fusion features according to the global fusion features of multiple consecutive time steps; performing hidden state calculation on the spatio-temporal fusion features through the LSTM layer to obtain a hidden state vector; mapping the hidden state vector into the obstacle recognition result through the output layer; determining a loss value according to the obstacle recognition result and the obstacle label; updating the CNN-LSTM hybrid neural network according to the loss value to obtain the trained obstacle recognition model.

4. The method of claim 1, wherein the method further comprises: The prediction of a target tire that will contact the obstacle according to the obstacle orientation specifically comprises: obtaining a current speed, a current acceleration, and a current steering angle of the current vehicle; predict a vehicle travel trajectory of the current vehicle according to the current vehicle speed, the current acceleration, and the current steering angle; determine a tire travel trajectory of each tire according to the tire distribution position of the current vehicle and the vehicle travel trajectory; predict the target tire that will contact the obstacle according to the obstacle position and the tire travel trajectory.

5. The method of claim 1, wherein the method further comprises: determine the tire protection operation for the target tire according to the obstacle type and the obstacle size, which specifically includes: when the obstacle type is a metal object and the obstacle size is less than or equal to a preset first size threshold, determine that the tire protection operation is to start an electromagnetic adsorption device and a powerful suction device around the tire; when the obstacle type is a non-metal object and the obstacle size is less than or equal to the first size threshold, determine that the tire protection operation is to start the powerful suction device around the tire; when the obstacle type is a road surface depression or a road surface protrusion and the obstacle size is less than or equal to a preset second size threshold, determine that the tire protection operation is to start a tire lifting function; when the obstacle type is a metal object or a non-metal object and the obstacle size is greater than the first size threshold, determine that the tire protection operation is to slow down and detour; when the obstacle type is a road surface depression or a road surface protrusion and the obstacle size is greater than the second size threshold, determine that the tire protection operation is to slow down and detour.

6. A machine vision-based tire protection method according to any one of claims 1 to 5, characterized in that, The tire protection method further includes the following steps: when the tire protection operation is to start the tire lifting function or slow down and detour, generate obstacle warning information according to the obstacle type, the obstacle size, and the obstacle position, and share the obstacle warning information to a navigation platform.

7. A machine vision-based tire protection method according to any one of claims 1 to 5, characterized in that, The tire protection method further includes the following steps: when the tire protection operation is to start the tire lifting function or slow down and detour, generate obstacle reporting information according to the obstacle type, the obstacle size, the obstacle position, and the road image information, and send the obstacle reporting information to a road administration system.

8. A machine vision based tire protection apparatus, characterized by, It includes: an information acquisition module configured to acquire road image information and radar detection information of a road ahead of a current vehicle; an obstacle identification module configured to input the road image information and the radar detection information to a pre-trained obstacle identification model to obtain an obstacle type, an obstacle size, and an obstacle position; a target tire determination module configured to predict a target tire that will contact the obstacle according to the obstacle position; a protection operation execution module configured to determine a tire protection operation for the target tire according to the obstacle type and the obstacle size, and execute the tire protection operation by the current vehicle; wherein a chassis around a tire of the current vehicle is provided with an electromagnetic adsorption device and a powerful suction device, an active suspension system of the current vehicle has a tire lifting function, and the tire protection operation includes at least one of starting the electromagnetic adsorption device around the tire, starting the powerful suction device around the tire, starting the tire lifting function, and slowing down and detouring.

9. An electronic device, comprising: It includes: at least one processor; at least one memory storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor realizes the machine vision-based tire protection method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the machine vision-based tire protection method according to any one of claims 1 to 7.

Images