Vehicle interior material identification method and device and robot

Abstract

The embodiment of the invention provides a vehicle interior material identification method and device and a robot, and relates to the technical field of vehicles. The method comprises the following steps: acquiring first spectral data of a to-be-identified material in the vehicle interior; extracting spectral features at a plurality of characteristic wavelengths and spectral texture features of a local region from the first spectral data; performing feature fusion processing on the spectral features and the local region spectral texture features to obtain fusion features; and inputting the fusion feature into the first model to obtain a material category identification result of the to-be-identified material. According to the method, the vehicle interior material identification accuracy and efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a method, device and robot for identifying vehicle interior materials. Background Technology

[0002] With the development of intelligent and high-end vehicles, the types of interior materials are becoming increasingly diverse, including genuine leather, Alcantara, wool blends, synthetic fabrics, slush-molded plastics, and wood grain veneers. These interior materials have significantly different requirements for cleaning processes.

[0003] Traditional vehicle interior cleaning operations rely heavily on manual experience. Cleaning personnel need to judge the materials of the vehicle interior by visual observation and manual touch, which can lead to problems such as misjudgment and low efficiency. Summary of the Invention

[0004] This application provides a method, apparatus, and robot for identifying vehicle interior materials, in order to improve the accuracy and efficiency of vehicle interior material identification.

[0005] In a first aspect, this application provides a method for identifying vehicle interior materials, comprising: acquiring first spectral data of a material to be identified in the vehicle interior; extracting spectral features at multiple characteristic wavelengths and local region spectral texture features from the first spectral data; performing feature fusion processing on the spectral features and local region spectral texture features to obtain fused features; and inputting the fused features into a first model to obtain the material category identification result of the material to be identified.

[0006] In one possible embodiment, the method is applied to a robot to obtain first spectral data of a material to be identified in the interior of a vehicle, including: collecting first spectral data of the material to be identified at sampling positions during robot movement; extracting spectral features and local spectral texture features at multiple characteristic wavelengths from the first spectral data, including: determining the local region where each characteristic wavelength is located by taking the sampling position corresponding to each characteristic wavelength as a reference point; and extracting texture features from the spectral data of the local region using a sliding window to obtain the local spectral texture features at the characteristic wavelengths.

[0007] In one possible embodiment, the first model is pre-trained using a training sample set, which includes interior material samples with at least one of different colors, wear levels, stain types, and contamination levels.

[0008] In one possible embodiment, the material category identification result of the material to be identified indicates that the material to be identified is a first material category and the confidence level of the first material category. The method further includes: if the confidence level of the first material category is greater than or equal to a first threshold, triggering the cleaning of the material to be identified of the first material category; if the confidence level of the first material category is less than the first threshold, acquiring an image of the material to be identified and using the image of the material to be identified to verify the material category of the material to be identified.

[0009] In one possible embodiment, after triggering the cleaning of the material to be identified in the first material category, the method further includes: cleaning the material to be identified in the first material category using cleaning parameters associated with the first material category, the cleaning parameters including a cleaning operation sequence, a cleaning tool corresponding to each cleaning operation in the cleaning operation sequence, a cleaning trajectory, and process parameters corresponding to each cleaning tool.

[0010] In one possible embodiment, after cleaning the material to be identified of a first material category, the method further includes: acquiring second spectral data of the cleaned material to be identified; determining, based on the second spectral data and the first spectral data, the spectral data differences of the material to be identified with respect to characteristic wavelengths of a specific stain type; and evaluating the cleaning effect of the material to be identified based on the spectral data differences.

[0011] In one possible embodiment, the method further includes: adjusting and updating the cleaning parameters using a preset cleaning parameter adjustment strategy when the cleaning effect is not up to standard; and re-cleaning the material to be identified using the updated cleaning parameters until the number of cleaning cycles reaches a second threshold.

[0012] In one possible embodiment, the method further includes: recording the material category and cleaning parameters of the material to be identified, and the updated cleaning parameters, in a knowledge base.

[0013] Secondly, this application provides a vehicle interior material identification device, which includes: an acquisition module for acquiring first spectral data of a material to be identified in the vehicle interior; an extraction module for extracting spectral features at multiple characteristic wavelengths and local region spectral texture features from the first spectral data; a fusion module for performing feature fusion processing on the spectral features and local region spectral texture features to obtain fused features; and an identification module for inputting the fused features into a first model to obtain the material category identification result of the material to be identified.

[0014] Thirdly, this application provides a robot comprising: a memory, a processor, and an execution component; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the method as described in any of the first aspects.

[0015] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method of any one of the first aspects.

[0017] In this embodiment, by acquiring the first spectral data of the material to be identified in the vehicle interior and extracting spectral features at multiple characteristic wavelengths and local spectral texture features from the first spectral data, automated material identification can be performed based on the absorption and reflection characteristics of the material to be identified to infrared light. This can be applied to vehicle interior material identification scenarios such as similar appearances, thereby improving the accuracy of vehicle interior material identification.

[0018] Furthermore, by extracting spectral features and local spectral texture features at multiple characteristic wavelengths from the first spectral data, efficient and accurate identification of vehicle interior materials can be achieved at more representative characteristic wavelengths. Single-point features and local region features at characteristic wavelengths are collected separately. By fusing the spectral features and local region spectral texture features, the aforementioned single-point features and local region features can be combined to obtain fused features. These fused features have better representativeness, facilitating accurate subsequent identification of the material category of the material to be identified.

[0019] By inputting the fused features into the first model, the material category identification result of the material to be identified is obtained. The material category identification result of the material to be identified can be automatically and quickly output, thereby improving the response speed of vehicle interior material identification. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a schematic diagram of the system architecture of the vehicle interior material identification method, device, and robot according to an embodiment of this application.

[0022] Figure 2 This is a flowchart of a vehicle interior material identification method according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of a vehicle interior material identification device according to an embodiment of this application.

[0024] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0026] Figure 1 This is a schematic diagram of the system architecture of the vehicle interior material identification method, device, and robot according to embodiments of this application.

[0027] like Figure 1 As shown, the system architecture includes robot 1 and vehicle 2.

[0028] For example, by triggering the vehicle interior material identification button set on the robot 1, relevant personnel can trigger the robot 1 to execute the vehicle interior material identification method of this application embodiment to identify the material category of the vehicle interior.

[0029] Apart from Figure 1 In addition to the system architecture shown, in another possible embodiment, the system architecture includes electronic devices, a robot, and a vehicle. The robot collects spectral data of the vehicle's interior trim and transmits the spectral data to the electronic devices. The electronic devices execute the vehicle interior trim material identification method of this application embodiment to identify the material category of the vehicle interior trim.

[0030] Electronic devices can also trigger robots to clean the vehicle interior based on the material category of the vehicle's interior.

[0031] Electronic devices can be computers, servers, etc. Both electronic devices and robots include processors and memory; robots also include actuators.

[0032] In one possible embodiment, the electronic device or robot's memory stores a knowledge base, which specifically stores vehicle model information, cleaning parameters, etc., to support the reuse of cleaning knowledge inside the vehicle.

[0033] Optionally, robots and electronic devices may also include communication components. Taking a robot comprising a processor, memory, actuator, and communication components as an example, the processor, memory, actuator, and communication components are connected via a bus.

[0034] In specific implementation, code is stored in memory, and the processor executes the code stored in memory to perform the method of the embodiments of this application. The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0035] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0036] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0037] The actuator is, for example, a robotic arm or an end effector, such as an adaptive gripper.

[0038] For example, the robot also includes a sensor array, which includes miniaturized near-infrared sensors or spectral imaging sensors for acquiring spectral data, with a wavelength range of [900, 1700] nm, which has significantly different spectral characteristics for different interior materials of organic polymers in the vehicle.

[0039] For example, the sensor array also includes an RGB color space camera and a distance sensor. The RGB camera is used to acquire images of the vehicle interior materials, which are then used to verify the material category of the vehicle interior materials identified through spectral data. The distance sensor is used to detect the distance at which the robot acquires spectral data of the vehicle interior materials, ensuring the consistency of the spectral data.

[0040] When cleaning vehicle interior materials, the robot's actuating components include various cleaning tools, such as high-frequency vibrating vacuum heads, soft-bristled roller brushes / scrapers, flexible polishing cloth discs, and metered sprayers. Alternatively, the actuating components may include an assembly integrating a micro-sprayer, a suction port, and interchangeable brush heads.

[0041] In related technology 1, personnel judge the materials of the vehicle's interior by visual observation and experience, and manually select cleaning tools and process parameters when cleaning the vehicle's interior. For example, a soft cloth is used to wipe leather materials, while a vacuum cleaner and foam cleaner are used for fabric materials. This method relies on manual experience and has problems such as low efficiency in identifying vehicle interior materials and low cleaning efficiency.

[0042] In related technology 2, a cleaning robot with a preset fixed process is used to clean the vehicle interior. However, this type of cleaning robot usually cannot identify the materials of the vehicle interior and simply uses preset cleaning parameters and preset cleaning paths. The cleaning parameters need to be set in advance, resulting in poor adaptability to different interior materials of the vehicle and poor cleaning effect.

[0043] In related technology 3, visual recognition technology is used to automatically identify vehicle interior materials. However, this method only uses features such as color and texture for identification, and its accuracy is low.

[0044] In summary, the relevant technologies have at least one of the following problems:

[0045] The accuracy of identifying vehicle interior materials is low. For example, it is impossible to accurately identify materials in the vehicle interior that have similar appearances but significantly different physical and chemical properties (such as imitation suede Alcantara and genuine leather), which leads to errors in the subsequent cleaning process.

[0046] Using a fixed cleaning process makes it impossible to dynamically adjust cleaning parameters (such as brush head speed, pressure, and cleaning agent type) according to the characteristics of vehicle interior materials, resulting in unstable cleaning effects or a high risk of material damage.

[0047] Without accurate assessment of cleaning effectiveness, continuous optimization of cleaning results is impossible.

[0048] Based on this, the vehicle interior material identification method, device and robot provided in this application, through the deep integration of spectral sensing technology and cleaning parameter adaptive mechanism, extract the essential characteristics of vehicle interior materials from complex spectral signals, and combine dynamic cleaning parameters to achieve automated and accurate identification and cleaning of vehicle interior materials.

[0049] This application is applicable to vehicle interior material identification and cleaning scenarios, such as vehicle interiors with irregular structures and different materials, such as seats, dashboards, and headliners.

[0050] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0051] Figure 2 This is a flowchart of a vehicle interior material identification method according to an embodiment of this application. The following explanation will use the example of a robot performing the vehicle interior material identification method.

[0052] like Figure 2 As shown, the vehicle interior material identification method includes steps S110 to S140.

[0053] S110. Obtain the first spectral data of the material to be identified in the vehicle interior.

[0054] For example, the robot sends infrared light of multiple wavelengths to the material to be identified in the vehicle interior through a miniaturized near-infrared sensor or a spectral imaging sensor. The light energy reflection intensity obtained after the infrared light shines on the surface of the material to be identified is the first spectral data.

[0055] For example, the first spectral data can be obtained through preprocessing, which includes dark current and white balance correction, and normalization. Dark current and white balance correction are used to reduce noise and ambient light effects in miniaturized near-infrared sensors or spectral imaging sensors. Normalization is used to reduce differences in light intensity and reflectance caused by surface curvature and micro-range variations of the material to be identified.

[0056] S120. Extract spectral features at multiple characteristic wavelengths and local region spectral texture features from the first spectral data.

[0057] The first spectral data represents multiple wavelengths of infrared light and the corresponding light energy reflection intensity for each wavelength. The wavelength with the best material characterization among these multiple wavelengths is the characteristic wavelength. These characteristic wavelengths include, for example, characteristic absorption wavelengths associated with cellulose, polyester, and protein amide bonds, to efficiently distinguish different interior materials.

[0058] The characteristic wavelengths and their corresponding light energy reflection intensities represent the spectral characteristics at multiple characteristic wavelengths. The texture characteristics exhibited by the spectral data of local areas of the material to be identified irradiated by multiple characteristic wavelengths represent the local spectral texture characteristics at multiple characteristic wavelengths.

[0059] S130. Perform feature fusion processing on spectral features and local region spectral texture features to obtain fused features.

[0060] For example, when the normalized spectral features and the local region spectral texture features have the same dimension, the spectral features and the local region spectral texture features can be weighted or concatenated to obtain fused features. That is, feature fusion processing includes weighted processing or concatenation processing.

[0061] S140. Input the fused features into the first model to obtain the material category identification result of the material to be identified.

[0062] For example, the first model can be a machine learning model or a deep learning model. Specifically, the first model can be at least one of support vector machines, random forest models, and convolutional neural networks.

[0063] In one possible embodiment, the first model is pre-trained using a training sample set, which includes interior material samples with at least one of different colors, wear levels, stain types, and contamination levels.

[0064] The intensity of light reflection, including the intensity of infrared light emitted onto and received by the material to be identified, is influenced by factors such as color, wear level, stain type, and pollution level. Different colors, wear levels, stain types, and pollution levels all affect the spectral characteristics and local spectral texture features. In this embodiment, the first model is pre-trained using interior material samples with at least one of different colors, wear levels, stain types, and pollution levels. This pre-training covers the dimensions of interior material variations in real-world scenarios, ensuring the accuracy and robustness of the first model in identifying the material category of the material to be identified.

[0065] In this embodiment, by acquiring the first spectral data of the material to be identified in the vehicle interior and extracting spectral features at multiple characteristic wavelengths and local spectral texture features from the first spectral data, automated material identification can be performed based on the absorption and reflection characteristics of the material to be identified to infrared light. This can be applied to vehicle interior material identification scenarios such as similar appearances, thereby improving the accuracy of vehicle interior material identification.

[0066] Furthermore, by extracting spectral features and local spectral texture features at multiple characteristic wavelengths from the first spectral data, efficient and accurate identification of vehicle interior materials can be achieved at more representative characteristic wavelengths. Single-point features and local region features at characteristic wavelengths are collected separately. By fusing the spectral features and local region spectral texture features, the aforementioned single-point features and local region features can be combined to obtain fused features. These fused features have better representativeness, facilitating accurate subsequent identification of the material category of the material to be identified.

[0067] By inputting the fused features into the first model, the material category identification result of the material to be identified is obtained. The material category identification result of the material to be identified can be automatically and quickly output, thereby improving the response speed of vehicle interior material identification.

[0068] In one possible embodiment, the method is applied to a robot, and step S110 of obtaining the first spectral data of the material to be identified in the vehicle interior includes step S111.

[0069] S111. Collect the first spectral data of the material to be identified at the sampling location during the robot's movement.

[0070] For example, the robot can preset the sampling position in advance, or when the robot moves to the area where the material to be identified is located, it can emit infrared light to the material to be identified at the sampling frequency and receive the light energy reflection intensity reflected by the material to be identified and record the sampling frequency as the sampling position.

[0071] Step S120, which involves extracting spectral features and local region spectral texture features from the first spectral data at multiple characteristic wavelengths, includes steps S121 to S122.

[0072] S121. Using the sampling position corresponding to each characteristic wavelength as a reference point, determine the local region where each characteristic wavelength is located.

[0073] At each sampling location, the robot emits infrared light towards the material to be identified at multiple wavelengths, including multiple characteristic wavelengths. For each characteristic wavelength, the sampling location can serve as a reference point, specifically the center point of the local region where the corresponding characteristic wavelength is located. This local region can be a rectangular or circular area.

[0074] S121. Use a sliding window to extract texture features from the spectral data of a local area to obtain the spectral texture features of the local area at the characteristic wavelength.

[0075] Taking a local region of 10*10 as an example, the sliding window can be 2*2 or 3*3, with a step size of 1, to ensure fine-grained texture feature extraction without losing texture details. Specifically, for each local region containing a feature wavelength and for each window, the robot can calculate the statistical characteristics of the spectral reflectance intensity within that window, such as mean, variance, entropy, extreme value difference, horizontal gradient, and vertical gradient. The statistical characteristics of the spectral reflectance intensity within each window are then concatenated to obtain the local spectral texture features at the feature wavelength.

[0076] In this embodiment, the first spectral data of the material to be identified is collected at sampling locations during robot movement, ensuring that the first spectral data takes into account both spectral and spatial dimensions. By using the sampling location corresponding to each characteristic wavelength as a reference point, the local region where each characteristic wavelength is located can be determined, expanding from a single point at the sampling location to the entire region where the sampling location is located. By using a sliding window to extract texture features from the spectral data of the local region, the resulting local spectral texture features at the characteristic wavelengths accurately reflect the characteristics of different interior materials.

[0077] Subsequently, the spectral features and local area spectral texture features are fused to obtain fused features. This allows for multi-scale spectral feature fusion based on single-point spectral features and local area spectral texture features, improving the accuracy of interior material identification. The embodiments of this application are applicable to accurately identifying easily confused interior materials.

[0078] For example, when the interior material is fabric, the spectral texture features of the fabric in local areas exhibit regular spatial variations due to the fiber weaving structure, while when the interior material is genuine leather, the spectral texture features of the fabric in local areas may show a more natural, non-uniform spectral texture.

[0079] In one possible embodiment, the material category identification result of the material to be identified indicates that the material to be identified belongs to a first material category and the confidence level of the first material category. That is, the first model is used to classify interior materials based on fused features, specifically outputting a label for each possible material category and the corresponding confidence level.

[0080] The method for identifying vehicle interior materials also includes steps S150 to S160.

[0081] S150. If the confidence level of the first material category is greater than or equal to the first threshold, the cleaning of the materials to be identified in the first material category is triggered.

[0082] S160. If the confidence level of the first material category is less than the first threshold, an image of the material to be identified is acquired and the material category of the material to be identified is verified using the image of the material to be identified.

[0083] For example, taking the confidence level of the first material as a range of [0,1], the first threshold value ranges from [0.7,0.95], and can specifically be 0.8.

[0084] For example, the robot can input an image of the material to be identified into a second model, which will then output the material category of the material. The second model can be a deep learning model for image classification, such as a convolutional neural network model or a YOLO model.

[0085] For example, if the material category of the material to be identified output by the second model is the first material category, that is, the output results of the second model and the first model are consistent, then the material to be identified can be determined to be the first material category. If the material category of the material to be identified output by the second model is not the first material category, that is, the output results of the second model and the first model are inconsistent, then a manual intervention request can be sent to request manual intervention to identify the material category of the material to be identified.

[0086] In one possible embodiment, if the confidence level of the first material category is less than a first threshold, the robot may adjust its position to acquire an image of the material to be identified from a different position and use the image of the material to be identified to verify the material category of the material to be identified.

[0087] In this embodiment, if the confidence level of the first material category is greater than or equal to the first threshold, it indicates that the material to be identified is credible as belonging to the first material category, and the next cleaning task can be directly performed, i.e., cleaning the material to be identified of the first material category. If the confidence level of the first material category is less than the first threshold, it indicates that the material to be identified is unreliable as belonging to the first material category, and in this case, to ensure the accuracy of the identification of the material to be identified, an image of the material to be identified is also used for verification.

[0088] In one possible embodiment, after triggering the cleaning of the material to be identified in the first material category in step S150, the vehicle interior material identification method further includes: cleaning the material to be identified in the first material category using cleaning parameters associated with the first material category, the cleaning parameters including a cleaning operation sequence, a cleaning tool corresponding to each cleaning operation in the cleaning operation sequence, a cleaning trajectory, and process parameters corresponding to each cleaning tool.

[0089] The first material category can be one of the following: genuine leather, synthetic leather, wool blend fabric, imitation suede, etc.

[0090] The cleaning sequence includes multiple cleaning operations, such as vacuuming, spraying, wiping, polishing, sweeping, and drying. Cleaning tools include vacuum heads, atomizing nozzles, soft-bristled roller brushes, drying heads, dusting cloths, and polishing cloth discs. Cleaning paths include reciprocating straight lines, spirals, and along seams. Process parameters include brush head speed, vibration frequency, vacuuming negative pressure, spray volume, cloth disc pressure, hot air temperature (if drying is required), cleaning fluid type code, and completion criteria for each cleaning operation (such as the visual assessment threshold for surface dust residue after vacuuming).

[0091] The following will use specific first material categories and corresponding cleaning parameters as examples.

[0092] For example, when the first material category is fabric or velvet, the cleaning parameters include the following cleaning operation sequence from cleaning operation a to cleaning operation d.

[0093] Among them, cleaning operation a indicates the use of a high-frequency vibrating vacuum head to cover the surface with constant pressure, high-frequency vibration (120Hz) to loosen deeply embedded dust, and strong negative pressure to suck it away.

[0094] Cleaning operation b indicates the use of a fine atomizing nozzle and a soft-bristled roller brush to first spray a small amount of neutral foam cleaner evenly; then the roller brush is used to circulate and scrub at a medium speed (e.g., 200 rpm), while the robot maintains a constant downward pressure of 5N.

[0095] The cleaning operation (C) indicator uses a high-powered wet and dry vacuum head to quickly remove residual foam and wastewater.

[0096] The cleaning operation d indicates the use of a ring-shaped hot air drying head, which uses low-temperature hot air (e.g., 40°C) to accelerate the evaporation of residual moisture.

[0097] For example, in the case where the first material category is leather, the cleaning parameters include the following cleaning operation sequence from cleaning operation a to cleaning operation d.

[0098] Among them, cleaning operation a indicates to use a soft dusting cloth to gently sweep across the surface and remove floating dust.

[0099] Cleaning operation b indicates the use of a metered spray and microfiber polishing cloth to spray a special leather cleaning solution; the cloth is then used to perform circumferential polishing at a constant low speed (e.g., 100 rpm) and light pressure (e.g., 3N) to absorb stains.

[0100] Cleaning operation c indicates the use of a dry polishing cloth for secondary dry polishing until the surface is shiny.

[0101] The cleaning operation d indicates the use of the polishing and maintenance agent application pad. Apply a small amount of maintenance agent and spread it evenly.

[0102] In this embodiment, by precisely matching the material categories of various interior materials with cleaning parameters, the problem of poor adaptability between fixed cleaning parameters and interior materials can be solved, ensuring that the cleaning process for interior materials is both efficient and safe. Furthermore, in this embodiment, the cleaning parameters for each material category include a cleaning operation sequence, a cleaning tool corresponding to each cleaning operation in the sequence, a cleaning trajectory, and process parameters corresponding to each cleaning tool, enabling fine cleaning of interior materials and improving the cleaning effect.

[0103] In one possible embodiment, if the material category of the material to be identified output by the second model is not the first material category, that is, the output results of the second model and the first model are inconsistent, the robot can use a preset conservative cleaning parameter to clean the material to be identified, such as dry vacuuming.

[0104] In one possible embodiment, the robot can acquire point cloud data of the vehicle's interior to digitally characterize the vehicle's interior in three dimensions. The robot can also map its cleaning trajectory onto the vehicle components containing the material to be identified within the point cloud data, ensuring that the cleaning tool contacts the surface at the optimal angle when cleaning the material. The robot can also employ constant force to clean the material, avoiding damage or poor cleaning results caused by unbalanced pressure.

[0105] In one possible embodiment, after cleaning the material to be identified in the first material category, the vehicle interior material identification method further includes steps S170 to S190.

[0106] S170. Obtain the second spectral data of the cleaned material to be identified.

[0107] The process of acquiring the second spectral data by the robot is similar to that of acquiring the first spectral data, and will not be described in detail here.

[0108] S180. Based on the second spectral data and the first spectral data, determine the spectral data differences of the material to be identified with respect to the characteristic wavelengths of a specific stain type.

[0109] For example, the difference between the light energy reflection intensity at each wavelength of the second spectral data and the first spectral data can be used as the spectral data difference of the material to be identified. Each stain type corresponds to its own characteristic wavelength, and the difference between the light energy reflection intensity at the characteristic wavelengths corresponding to various stain types in the spectral data difference of the material to be identified is the spectral data difference of the material to be identified with respect to the characteristic wavelengths of a specific stain type.

[0110] S190. Evaluate the cleaning effect of the material to be identified based on the differences in spectral data.

[0111] For example, by comparing the spectral data differences of the material to be identified with respect to the characteristic wavelengths of a specific stain type, it is possible to determine whether and to what extent the absorption peak of a specific stain type at the characteristic wavelength is reduced before and after cleaning, so as to quantitatively evaluate the cleaning effect of the material to be identified.

[0112] In this embodiment, the above-described operation allows for automated evaluation of the cleaning effect of the material to be identified by re-scanning its spectral data after a single cleaning, and then reusing the spectral data differences before and after cleaning. Furthermore, based on the second and first spectral data, the spectral data differences of the material to be identified with respect to the characteristic wavelengths of specific stain types are determined, enabling a refined evaluation of the cleaning effect at the granularity of specific stain types.

[0113] In one possible embodiment, the vehicle interior material identification method further includes: adjusting and updating the cleaning parameters using a preset cleaning parameter adjustment strategy when the cleaning effect is not up to standard; and re-cleaning the material to be identified using the updated cleaning parameters until the number of cleaning times reaches a second threshold.

[0114] Whether the cleaning effect meets the standard can be determined by whether the absorption peak of the stain type at the characteristic wavelength is reduced and the degree of reduction. If the stain types of the material to be identified are reduced by more than 80% before and after cleaning, and the absorption peak of each stain type at the characteristic wavelength is reduced by more than 60%, the cleaning effect meets the standard; otherwise, the cleaning effect does not meet the standard.

[0115] For example, the preset cleaning parameter adjustment strategy indicates at least one of the following: increasing the amount of cleaning agent used, extending the cleaning trajectory, or increasing the pressure.

[0116] For example, the second threshold ranges from [2,4], and for example, the value of the second threshold is 2. Cleaning is terminated when the second threshold is reached.

[0117] In this embodiment, when the cleaning effect is not up to standard, the cleaning parameters can be adjusted and updated using a preset cleaning parameter adjustment strategy, thereby enabling closed-loop processing of the cleaning parameters. The material to be identified is then cleaned again using the updated cleaning parameters until the number of cleaning cycles reaches the second threshold. This allows for further cleaning of the material to be identified with more effective cleaning parameters, ensuring the cleaning effect of the material to be identified.

[0118] In one possible embodiment, the vehicle interior material identification method further includes: recording the material category and cleaning parameters of the material to be identified, as well as the updated cleaning parameters, in a knowledge base.

[0119] In this embodiment, by recording the material category and cleaning parameters of the material to be identified, as well as the updated cleaning parameters, in the knowledge base, the relevant knowledge from the current effective cleaning can be reused, improving the reliability of the robot in cleaning interior materials. Furthermore, the knowledge base can be updated to ensure that the robot's knowledge regarding interior material cleaning is iteratively updated, supporting the robot's continuous learning capabilities.

[0120] Of course, in addition to cleaning-related knowledge, the knowledge related to interior material identification can also be recorded and updated during the interior material identification stage.

[0121] In summary, the vehicle interior material identification method of this application introduces spectral sensing technology into the process of robotic identification and cleaning of vehicle interior materials, achieving a leap from "seeing the color" to "perceiving the essence of the interior material." By constructing a spectral fingerprint of the material to be identified (such as spectral features and local spectral texture features) and a complete technical chain based on this spectral fingerprint for identification, a refined cleaning knowledge base, and adaptive cleaning and feedback, the professionalism, safety, and automation level of vehicle interior material identification and cleaning are ensured.

[0122] Figure 3 This is a schematic diagram of the vehicle interior material identification device according to an embodiment of this application. Figure 3 As shown, the vehicle interior material identification device provided in this application embodiment includes: an acquisition module 210, an extraction module 220, a fusion module 230, and an identification module 240.

[0123] The acquisition module 210 is used to acquire the first spectral data of the material to be identified in the vehicle interior.

[0124] Extraction module 220 is used to extract spectral features at multiple characteristic wavelengths and local region spectral texture features from the first spectral data.

[0125] The fusion module 230 is used to perform feature fusion processing on spectral features and local region spectral texture features to obtain fused features.

[0126] The identification module 240 is used to input the fused features into the first model to obtain the material category identification result of the material to be identified.

[0127] In one possible embodiment, the device method is applied to a robot. The acquisition module is specifically used to collect first spectral data of the material to be identified at the sampling position during the robot's movement. The extraction module includes: a local region determination submodule, used to determine the local region where each feature wavelength is located, with the sampling position corresponding to each feature wavelength as a reference point; and a texture feature extraction submodule, used to extract texture features from the spectral data of the local region using a sliding window to obtain the local region spectral texture features at the feature wavelength.

[0128] In one possible embodiment, the first model is pre-trained using a training sample set, which includes interior material samples with at least one of different colors, wear levels, stain types, and contamination levels.

[0129] In one possible embodiment, the material category identification result of the material to be identified indicates that the material to be identified is a first material category and the confidence level of the first material category. The device further includes: a triggering module, used to trigger cleaning of the material to be identified of the first material category when the confidence level of the first material category is greater than or equal to a first threshold; and a verification module, used to acquire an image of the material to be identified and verify the material category of the material to be identified using the image of the material to be identified when the confidence level of the first material category is less than the first threshold.

[0130] In one possible embodiment, the device further includes a cleaning module for cleaning a material to be identified in the first material category using cleaning parameters associated with the first material category, the cleaning parameters including a cleaning operation sequence, a cleaning tool corresponding to each cleaning operation in the cleaning operation sequence, a cleaning trajectory, and process parameters corresponding to each cleaning tool.

[0131] In one possible embodiment, the acquisition module is further configured to acquire second spectral data of the cleaned material to be identified; the device further includes: a determination module, configured to determine the spectral data differences of the material to be identified with respect to characteristic wavelengths of a specific stain type based on the second spectral data and the first spectral data; and an evaluation module, configured to evaluate the cleaning effect of the material to be identified based on the spectral data differences.

[0132] In one possible embodiment, the device further includes: an adjustment module for adjusting and updating cleaning parameters using a preset cleaning parameter adjustment strategy when the cleaning effect is not up to standard; and a re-cleaning module for re-cleaning the material to be identified using the updated cleaning parameters until the number of cleaning cycles reaches a second threshold.

[0133] In one possible embodiment, the device further includes a recording module for recording the material category and cleaning parameters of the material to be identified, as well as the updated cleaning parameters, in a knowledge base.

[0134] This application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the above-described method embodiments.

[0135] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0136] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0137] This application provides a computer program product, including a computer program that, when executed by a processor, implements the methods provided in any of the embodiments described above.

[0138] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0139] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0140] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0141] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0142] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0143] If the integrated unit / module is implemented as a software program module and sold or used as an independent financial product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software financial product. This computer software financial product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0144] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0145] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0146] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for identifying vehicle interior materials, characterized in that, The method includes: Acquire the first spectral data of the material to be identified in the vehicle interior; Extract spectral features at multiple characteristic wavelengths and local region spectral texture features from the first spectral data; The spectral features and the local region spectral texture features are fused to obtain the fused features. The fused features are input into the first model to obtain the material category identification result of the material to be identified.

2. The method according to claim 1, characterized in that, The method is applied to robots, and the acquisition of the first spectral data of the material to be identified in the vehicle interior includes: The first spectral data of the material to be identified is collected at the sampling location during the robot's movement. The extraction of spectral features and local region spectral texture features at multiple characteristic wavelengths from the first spectral data includes: Using the sampling position corresponding to each characteristic wavelength as a reference point, determine the local region where each characteristic wavelength is located; The spectral data of the local region is extracted using a sliding window to obtain the spectral texture features of the local region at the characteristic wavelength.

3. The method according to claim 1, characterized in that, The first model is pre-trained using a training sample set, which includes interior material samples with at least one of different colors, wear levels, stain types, and pollution levels.

4. The method according to any one of claims 1-3, characterized in that, The material category identification result of the material to be identified indicates the material to be identified as a first material category and the confidence level of the first material category. The method further includes: If the confidence level of the first material category is greater than or equal to the first threshold, the cleaning of the material to be identified in the first material category is triggered. If the confidence level of the first material category is less than the first threshold, an image of the material to be identified is acquired and the material category of the material to be identified is verified using the image of the material to be identified.

5. The method according to claim 4, characterized in that, After triggering the cleaning of the material to be identified in the first material category, the method further includes: The material to be identified in the first material category is cleaned using cleaning parameters associated with the first material category. The cleaning parameters include a cleaning operation sequence, a cleaning tool corresponding to each cleaning operation in the cleaning operation sequence, a cleaning trajectory, and process parameters corresponding to each cleaning tool.

6. The method according to claim 5, characterized in that, After cleaning the material to be identified in the first material category, the method further includes: Acquire the second spectral data of the cleaned material to be identified; Based on the second spectral data and the first spectral data, determine the spectral data differences of the material to be identified with respect to the characteristic wavelengths of a specific stain type; The cleaning effect of the material to be identified is evaluated based on the differences in the spectral data.

7. The method according to claim 6, characterized in that, The method further includes: If the cleaning effect is not up to standard, the cleaning parameters are adjusted and updated using a preset cleaning parameter adjustment strategy. The material to be identified is re-cleaned using the updated cleaning parameters until the number of cleaning cycles reaches the second threshold.

8. The method according to claim 7, characterized in that, The method further includes: The material category of the material to be identified, the cleaning parameters, and the updated cleaning parameters are recorded in the knowledge base.

9. A vehicle interior material identification device, characterized in that, The device includes: The acquisition module is used to acquire the first spectral data of the materials to be identified in the vehicle interior. The extraction module is used to extract spectral features at multiple characteristic wavelengths and local region spectral texture features from the first spectral data; The fusion module is used to perform feature fusion processing on the spectral features and the local region spectral texture features to obtain fused features; The identification module is used to input the fused features into the first model to obtain the material category identification result of the material to be identified.

10. A robot, characterized in that, include: Memory, processor, and execution unit; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

Images