An in-vehicle safety risk detection method and device, a vehicle, and a storage medium

Abstract

The application provides an in-vehicle safety risk detection method and device, a vehicle and a storage medium. The method obtains a preset model prompt word, a current in-vehicle image of a to-be-detected vehicle, a safe in-vehicle image and current in-vehicle environment information. If the current in-vehicle image is different from the safe in-vehicle image, the current in-vehicle image is input into an image processing model to obtain a current natural language description for the current in-vehicle image. A current in-vehicle safety risk detection description sentence is generated based on the current natural language description, the current in-vehicle environment information and the preset model prompt word. The current in-vehicle safety risk detection description sentence is input into an inference large language model to speculate whether the to-be-detected vehicle has an in-vehicle safety risk, and an in-vehicle safety risk detection result of the to-be-detected vehicle is obtained. The safety recognition coverage is effectively improved, the detection accuracy is improved, the detection reliability is improved regardless of whether a pre-established dangerous goods library is comprehensive, and driving safety is improved.

Description

Technical Field

[0001] This application relates to the field of automotive safety monitoring technology, and in particular to a method, device, vehicle, and storage medium for detecting in-vehicle safety risks. Background Technology

[0002] Many car owners like to personalize their vehicle interiors, such as adding steering wheel kits or modifying the steering wheel material, or placing hard or dangerous items on the center console. This significantly increases driving risks.

[0003] In related technologies, the monitoring of hazardous materials in the cabin often involves identifying items within the cabin and then searching a pre-established hazardous materials database for their presence. This process determines whether the identified item is hazardous and thus assesses the potential safety risks within the cabin. However, this method is limited by the accuracy and comprehensiveness of the hazardous materials database, leading to missed or false detections. Consequently, the accuracy and reliability of monitoring in-vehicle safety risks are low, potentially impacting driving safety. Summary of the Invention

[0004] This application provides a method, device, vehicle, and storage medium for detecting in-vehicle safety risks, in order to solve the technical problems in related technologies where the accuracy and reliability of in-vehicle safety risk monitoring are not high, which may affect driving safety.

[0005] This application provides a method for detecting in-vehicle safety risks. The method includes: acquiring preset model prompts, as well as the current in-vehicle image, the safe in-vehicle image, and the current in-vehicle environment information of the vehicle to be detected; if the current in-vehicle image is different from the safe in-vehicle image, inputting the current in-vehicle image into an image processing model to obtain a current natural language description of the current in-vehicle image; generating a current in-vehicle safety risk detection description statement based on the current natural language description, the current in-vehicle environment information, and the preset model prompts; inputting the current in-vehicle safety risk detection description statement into an inference language model to speculate on whether the vehicle to be detected has in-vehicle safety risks, and obtaining the in-vehicle safety risk detection result of the vehicle to be detected.

[0006] The method provided in the above embodiments obtains preset model prompts, as well as the current in-vehicle image, safe in-vehicle image, and current in-vehicle environment information of the vehicle to be detected. If the current in-vehicle image differs from the safe in-vehicle image, the current in-vehicle image is input into an image processing model to obtain a current natural language description for the current in-vehicle image. Based on the current natural language description, the current in-vehicle environment information, and the preset model prompts, a current in-vehicle safety risk detection description statement is generated. The current in-vehicle safety risk detection description statement is input into a large-scale inference language model to infer whether there is an in-vehicle safety risk in the vehicle to be detected, thereby obtaining the vehicle's in-vehicle safety risk. The interior safety risk detection results enable the identification of potential safety hazards inside the vehicle, improving passenger compartment safety. Leveraging the extensive knowledge reasoning capabilities of the reasoning language model, the system effectively enhances the coverage of safety identification, demonstrating a certain degree of identifiability for various safety hazards. By combining in-vehicle environmental information with the item information provided by the current natural language description, the system comprehensively judges whether there are any safety hazards, resulting in higher accuracy in interior safety risk detection. This method is no longer affected by the comprehensiveness of the pre-established hazardous materials database, improving the reliability of the detection and effectively enhancing driving safety.

[0007] In one embodiment of this application, before acquiring the current in-vehicle image of the vehicle to be detected, the method further includes: acquiring image acquisition device parameters of the safe in-vehicle image of the vehicle to be detected; adjusting the parameters of the image acquisition device of the vehicle to be detected based on the image acquisition device parameters; and acquiring the current in-vehicle image using the adjusted image acquisition device. By maintaining consistency between the device parameters at the time of acquisition of the safe in-vehicle image and the current in-vehicle image, misjudgments caused by device parameter issues can be minimized, thereby improving the detection accuracy and efficiency of the method.

[0008] In one embodiment of this application, acquiring a safe interior image of the vehicle to be detected includes, under safe conditions, acquiring images of multiple areas to be detected inside the vehicle using at least one image acquisition device installed on the vehicle to be detected, thereby obtaining multiple safe interior images; acquiring a current interior image of the vehicle to be detected includes, acquiring images of the areas to be detected inside the vehicle using at least one image acquisition device, thereby obtaining one or more current interior images, wherein the image acquisition device parameters of the image acquisition device are consistent with the image acquisition device parameters when acquiring the corresponding safe interior images; comparing the current interior image acquired by the same image acquisition device with the safe interior image, and if the current interior image is different from the safe interior image, triggering the step of inputting the current interior image into an image processing model. By acquiring safe interior images under safe conditions and then acquiring current interior images, the time and computing power for identifying items inside the vehicle can be saved. By identifying the different targets in the two images and then making targeted judgments, the detection efficiency can be improved.

[0009] In one embodiment of this application, the method further includes: if the current in-vehicle image is the same as the safe in-vehicle image, then no further assessment of the in-vehicle safety risk of the area to be detected corresponding to the current in-vehicle image is performed. If there is no change in the in-vehicle area, that is, the two images are the same, then no detection is performed, saving computing resources and improving the speed of determining the detection result.

[0010] In one embodiment of this application, the current in-vehicle environment information includes at least one of the following: in-vehicle temperature, in-vehicle humidity, in-vehicle fine particulate matter concentration, and in-vehicle inhalable particulate matter concentration.

[0011] In one embodiment of this application, after acquiring the current in-vehicle image and the safe in-vehicle image of the vehicle to be detected, the method further includes: matching the current in-vehicle image and the safe in-vehicle image based on the device identifier and image acquisition device parameters of the image acquisition device at the time of image acquisition, to obtain one or more sets of image pairs; The current in-vehicle image and the safe in-vehicle image are input into a preset image feature extraction model to obtain the current image feature vector and the safe image feature vector. Similarity is calculated for the current image feature vector and the safe image feature vector of the same image pair. If the similarity is less than a preset similarity threshold, the current in-vehicle image and the safe in-vehicle image of the same image pair are determined to be different. By first combining the images to obtain image pairs and then performing similarity comparison, the comparison efficiency can be greatly improved, thereby improving the execution efficiency of detection and the speed of early warning.

[0012] In one embodiment of this application, after obtaining the in-vehicle safety risk detection result of the vehicle to be detected, the method further includes: if the in-vehicle safety risk detection result is dangerous, generating alarm information based on the current in-vehicle image of the input image processing model and the inference analysis content output by the inference big data model; and feeding back the alarm information to a preset prompt terminal through an in-vehicle communication device. The alarm information can be displayed using both text and images based on the current in-vehicle image of the area where the abnormal item appears and the inference analysis content, improving the intuitiveness of the information.

[0013] In one embodiment of this application, after the alarm information is fed back to a preset prompt terminal via the in-vehicle communication device, the method further includes: obtaining a manual confirmation instruction; responding to the manual confirmation instruction, adjusting at least a portion of the image acquisition equipment to capture real-time images of the vehicle to be detected; and feeding back the real-time images to the preset prompt terminal via the in-vehicle communication device. When the user confirms the need to view the situation inside the vehicle in real time, real-time images of the vehicle interior are provided for the user to view in real time, understand the real-time status inside the vehicle, and provide a more accurate basis for risk management for the user or other relevant personnel.

[0014] This application also provides an in-vehicle safety risk detection device, the device comprising: an acquisition module, used to acquire preset model prompt words, and current in-vehicle image, safe in-vehicle image, and current in-vehicle environment information of the vehicle to be detected; an image description module, used to input the current in-vehicle image into an image processing model to obtain a current natural language description of the current in-vehicle image if the current in-vehicle image is different from the safe in-vehicle image; a sentence generation module, used to generate a current in-vehicle safety risk detection description sentence based on the current natural language description, the current in-vehicle environment information, and the preset model prompt words; and a detection module, used to input the current in-vehicle safety risk detection description sentence into an inference large language model to speculate whether the vehicle to be detected has in-vehicle safety risks, and obtain the in-vehicle safety risk detection result of the vehicle to be detected.

[0015] This application also provides a vehicle, which includes an image acquisition device, an information acquisition module, an image comparison module, an image description module, a statement generation module, a sending module, and a receiving module, wherein: the image acquisition device is used to acquire a current in-vehicle image and a safe in-vehicle image of the vehicle to be detected; the information acquisition module is used to acquire preset model prompts and current in-vehicle environment information; the image comparison module is used to compare the current in-vehicle image with the safe in-vehicle image; the image description module is used to input the current in-vehicle image into an image processing model to obtain a current natural language description for the current in-vehicle image if the current in-vehicle image is different from the safe in-vehicle image; the statement generation module is used to generate a current in-vehicle safety risk detection description statement based on the current natural language description, the current in-vehicle environment information, and the preset model prompts; the sending module is used to send the current in-vehicle safety risk detection description statement to an inference language model, and use the inference language model to infer whether there is an in-vehicle safety risk in the vehicle to be detected, thereby obtaining the in-vehicle safety risk detection result of the vehicle to be detected; the receiving module is used to receive the in-vehicle safety risk detection result.

[0016] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0017] This application also provides an electronic device, including: a memory storing a computer program thereon; and a processor for executing the computer program in the memory to implement the steps of the method described in any of the above embodiments.

[0018] The beneficial effects of this application are as follows: This application proposes a method, device, vehicle, and storage medium for detecting in-vehicle safety risks. The method acquires preset model prompts, as well as the current in-vehicle image, a safe in-vehicle image, and current in-vehicle environment information of the vehicle to be detected. If the current in-vehicle image differs from the safe in-vehicle image, the current in-vehicle image is input into an image processing model to obtain a current natural language description of the current in-vehicle image. Based on the current natural language description, the current in-vehicle environment information, and the preset model prompts, a current in-vehicle safety risk detection description statement is generated. This description statement is then input into a large-scale inference language model to determine whether the vehicle to be detected has in-vehicle safety risks. Risk estimation yields in-vehicle safety risk detection results for the vehicle under test, enabling the identification of potential safety hazards and improving passenger compartment safety. Leveraging the extensive knowledge reasoning capabilities of the reasoning language model, the identification of in-vehicle safety hazards is effectively enhanced, improving safety identification coverage and demonstrating a certain degree of identifiability for various safety hazards. Combining in-vehicle environmental information with item information provided by current natural language descriptions to comprehensively determine the existence of safety hazards further increases the accuracy of in-vehicle safety risk detection. This method is no longer affected by the comprehensiveness of a pre-established hazardous materials database, improving detection reliability and effectively enhancing driving safety. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] In the attached diagram: Figure 1 This is a schematic diagram illustrating an application scenario of an in-vehicle safety risk detection method provided in an embodiment of this application. Figure 2 A schematic flowchart of an in-vehicle safety risk detection method provided in an embodiment of this application; Figure 3 A schematic flowchart illustrating the comparison between a current in-vehicle image and a safe in-vehicle image provided in an embodiment of this application; Figure 4 A schematic flowchart illustrating a specific method for detecting in-vehicle safety risks according to an embodiment of this application; Figure 5 A schematic diagram of the structure of an in-vehicle safety risk detection device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0022] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0023] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0024] The inventors discovered that in related technologies, the monitoring of hazardous materials in the cabin often involves establishing a hazardous materials database and associating identified objects with this database to determine whether an object is hazardous. This method limits the number of hazardous materials and requires personnel to add new hazardous materials data to the database. Furthermore, the monitoring of hazards does not consider the relationship between objects and their environment. For example, the danger level of placing a lighter inside a car in cold winter is different from that in hot summer weather. Additionally, technologies cannot detect vehicle modifications, such as replacing the steering wheel with a slippery or fragile material, thus failing to identify such hazards. Based on this, a vehicle interior safety risk detection method is proposed. This method uses multi-source image comparison and large-scale model judgment to intelligently monitor in-vehicle cabin safety risks. For example, a baseline image database of vehicle manufacturing images can be established by collecting safe in-vehicle images. Subsequent use of the vehicle allows for the collection of multi-view in-vehicle images (current in-vehicle images) and environmental information (current in-vehicle environmental information). The method analyzes the degree of danger in changing areas within the vehicle in real time and promptly feeds the results back to the owner, effectively preventing accidents.

[0025] Please see Figure 1 , Figure 1This is a schematic diagram illustrating an application scenario of an in-vehicle safety risk detection method provided in an embodiment of this application. For example... Figure 1 As shown, this application scenario may include a vehicle 110, a cloud server 120, and a user terminal 130. One or more image acquisition devices are deployed inside the vehicle, which can acquire images of the corresponding areas to be detected inside the vehicle when needed by the user. The vehicle is also equipped with an in-vehicle communication device, through which the data acquired by the image acquisition devices can be transmitted to the cloud server and / or the user terminal.

[0026] Vehicle 110 can be any of the following: traditional gasoline vehicle, hybrid vehicle, pure electric vehicle, plug-in hybrid vehicle, hydrogen fuel cell vehicle, range-extended vehicle, etc. This application does not specifically limit the type of vehicle 110. The specific implementation method of the cloud server can be implemented in a way known to those skilled in the art, and is not limited here.

[0027] User terminal 130 can be a single terminal device, such as a laptop, desktop computer, mobile phone, tablet, wristband with information display function, watch with information display function, etc., or it can be a combination of multiple terminal devices mentioned above. This application does not specifically limit the specific form and type of user terminal 130.

[0028] Image acquisition devices can capture both safe interior images and current interior images of the area to be detected inside the vehicle. The image acquisition parameters of the devices must remain consistent during both acquisitions. The safe interior images and current interior images are then compared. If they differ, the current safe interior image is input into the image processing model to obtain a natural language description. This description is then combined with current in-vehicle environmental information and preset model prompts to generate a current in-vehicle safety risk detection description. This description is then input into a large-scale inference language model to infer the existence of in-vehicle safety risks, resulting in a vehicle safety risk detection result. This method, which analyzes images of areas that have changed, saves computational resources and allows for a comprehensive assessment of potential safety hazards by combining current in-vehicle environmental information, making in-vehicle safety risk detection more reliable and accurate. If the in-vehicle safety risk detection result is dangerous, a corresponding alarm message can be generated and sent to the user terminal via the in-vehicle communication device for notification.

[0029] As an example, the system could send the captured safe in-vehicle images and the current in-vehicle images to a cloud server via an in-vehicle communication device. The cloud server would then compare the images. If they differ, an image processing model deployed on the cloud server would be used to obtain a natural language description, which would then generate a description of the current in-vehicle safety risks. A large inference language model deployed on the cloud server would then be used to infer whether there are any in-vehicle safety risks. The in-vehicle safety risk detection results would be fed back to the vehicle, and then sent to the user terminal via the vehicle's in-vehicle communication device.

[0030] As another example, image processing models and inference language models can be deployed on the vehicle side, and the in-vehicle safety risk detection results can be obtained locally on the vehicle side. Then, the in-vehicle safety risk detection results can be sent to the user terminal through the in-vehicle communication device. In this case, the cloud server is not a necessary component of the application scenario.

[0031] As another example, after obtaining the in-vehicle safety risk detection results from the cloud server, the results can be sent directly to the user's terminal via the cloud server.

[0032] It should be noted that the collection and transmission of in-vehicle images and other data involving user privacy were all achieved using reasonable and legal technical means with the authorization of the relevant parties. The processing of sensitive user data will also comply with the requirements of relevant laws and regulations, which will not be elaborated upon further below.

[0033] It should be noted that the above scenario is only an example of an application scenario provided by the embodiments of this application. The embodiments of this application do not limit the actual form of the various devices included in the scenario. In the specific application of the solution, it can be set according to actual needs. Please see Figure 2 , Figure 2 A flowchart illustrating a method for detecting in-vehicle safety risks according to an embodiment of this application is shown below. Figure 2 As shown, the method includes the following steps: Step S210: Obtain preset model prompt words, as well as the current in-vehicle image, safe in-vehicle image, and current in-vehicle environment information of the vehicle to be detected.

[0034] The preset model prompts can be pre-configured by those skilled in the art based on the features of the large language model used in subsequent reasoning. For example, "You are a driver with 20 years of experience and have extensive experience in identifying safety hazards inside vehicles. Please determine whether there are any safety hazards in the passenger compartment based on the following description. If there are, please briefly describe them." When setting preset model prompts, qualifying words for safety hazards that the model is prone to misinterpretation can also be provided to further improve the accuracy of the model's reasoning results. Preset model prompts can also be set in conjunction with user preferences. For example, the system's preset model prompts can be provided to users (vehicle users, etc.) who can modify them as needed, adding their own preferences. Alternatively, preset model prompts can be set based on vehicle model, user gender, age, occupation, etc. The selection is based on the vehicle model to be detected and the current user information.

[0035] Current in-vehicle environment information can also be understood as current in-vehicle environment data. As an example, current environmental information can be obtained through vehicle body sensors, including but not limited to: in-vehicle temperature, in-vehicle humidity, in-vehicle PM2.5 (fine particulate matter) concentration, and in-vehicle PM10 (inhalable particulate matter) concentration, among others. The specific parameters of the current in-vehicle environment information can be added or removed based on the needs of those skilled in the art. The above is only an example and is not a requirement to collect parameters such as in-vehicle temperature in order to detect in-vehicle safety risks.

[0036] Both the current in-vehicle image and the safe in-vehicle image are images of the area to be detected inside the vehicle. As an example, the number of images in the current in-vehicle image and the safe in-vehicle image can be the same or different. As an example, the number of images in the current in-vehicle image can be less than or equal to the number of images in the safe in-vehicle image. The number of images in the safe in-vehicle image can be one or more.

[0037] In one embodiment, before acquiring the current in-vehicle image of the vehicle to be detected, the method further includes: acquiring image acquisition device parameters of the safe in-vehicle image of the vehicle to be detected; adjusting the parameters of the image acquisition device of the vehicle to be detected based on the image acquisition device parameters; and acquiring the current in-vehicle image using the adjusted image acquisition device.

[0038] To simplify subsequent image comparison and save computational resources, the same image acquisition device with identical parameters can be used to acquire both the current in-vehicle image and the safe in-vehicle image. Alternatively, if requested by those skilled in the art, different image acquisition devices or different parameters can be used. This approach involves first identifying the marked objects and then comparing them, or transforming the image coordinate system to identify the objects in both images and then comparing them, or other technical solutions known to those skilled in the art.

[0039] For example, one or more image acquisition devices can be installed inside the vehicle. Since the image acquisition devices inside the vehicle usually do not change position frequently, only the parameters of the image acquisition devices need to be adjusted to achieve the acquisition of the current in-vehicle image at the same angle, eliminating the need for subsequent image coordinate system transformation and other processing, and allowing for direct comparison.

[0040] An image acquisition device can be used to acquire an image of a region to be detected. Of course, by adjusting the angle of the image acquisition device, images of multiple regions to be detected can be acquired. The specific choice can be made by those skilled in the art as needed.

[0041] Image acquisition devices can be cameras or other devices with image acquisition functions known to those skilled in the art, such as cameras or webcams.

[0042] To facilitate subsequent image comparison, the current in-vehicle image and the safe in-vehicle image can be associated and stored based on the image acquisition device and its parameters during image acquisition. Taking a camera as the image acquisition device, the initial image of the passenger compartment as the safe in-vehicle image, and the camera angle as the image acquisition device parameters as an example, the original in-vehicle image is acquired at the initial stage of vehicle manufacturing or purchase as the safe in-vehicle image, establishing a vehicle manufacturing baseline image library. At this time, no decorations have been added to the vehicle, and no modifications have been made; it is the initial state of the passenger compartment. Images need to be acquired from all directions inside the vehicle. During the acquisition process, the camera angle and image ID are recorded. After acquisition, the original image is stored in the database. As an example, the safe in-vehicle image acquisition record table is shown in Table 1.

[0043] Table 1. Safety In-Vehicle Image Acquisition Record Table

[0044] After a vehicle has been purchased for some time, the interior condition may change, which may lead to certain safety hazards. For example, placing a lighter on the dashboard in hot weather, placing fragile items, knives, or other sharp objects in inappropriate locations, modifying the steering wheel or dashboard, or improperly modifying the center console screen.

[0045] Safety risk assessment is conducted by collecting data from the passenger compartment while the vehicle is unmanned. The acquired data consists of two parts. The first part is the current in-vehicle image data. Based on Table 1, the camera angle is adjusted to capture the current image at the corresponding location, forming the current in-vehicle image acquisition record table, as shown in Table 2. It should be noted that the image format here is only an example, and the specific image format can be selected by those skilled in the art as needed.

[0046] Table 2 Current In-Vehicle Image Acquisition Record Table

[0047] By using this corresponding storage method, image pairs can be quickly found and compared when image comparison is needed later.

[0048] In one embodiment, after acquiring the current in-vehicle image and the safe in-vehicle image of the vehicle to be detected, the method further includes: matching the current in-vehicle image and the safe in-vehicle image based on the device identifier and parameters of the image acquisition device at the time of image acquisition, to obtain one or more image pairs; inputting the current in-vehicle image and the safe in-vehicle image into a preset image feature extraction model to obtain the current image feature vector and the safe image feature vector; calculating the similarity between the current image feature vector and the safe image feature vector of the same image pair; if the similarity is less than a preset similarity threshold, then it is determined that the current in-vehicle image and the safe in-vehicle image of the same image pair are different. If the similarity is greater than or equal to the preset similarity threshold, then it is determined that the current in-vehicle image and the safe in-vehicle image of the same image pair are the same, and no processing is required for the current in-vehicle image in this image pair.

[0049] The area inside the vehicle is relatively large. If a single image could encompass the entire interior, the method provided in this embodiment could be implemented. However, this approach would result in high computational costs for subsequent steps. In this case, multiple images can be acquired from different areas. Each image is recorded with the device identifier and parameters of the image acquisition device. Subsequently, the images can be grouped based on the device identifier and parameters of the acquired images for targeted comparison. This greatly improves comparison efficiency and saves computational power.

[0050] Step S220: If the current in-vehicle image is different from the safe in-vehicle image, input the current in-vehicle image into the image processing model to obtain the current natural language description for the current in-vehicle image.

[0051] After obtaining the current in-vehicle image and the safe in-vehicle image, if there is only one image for each, they can be compared directly. If there are multiple safe in-vehicle images and one or more current in-vehicle images, then the comparison needs to be performed according to the corresponding images.

[0052] As an example, the current in-vehicle image and the safe in-vehicle image in the above embodiments can be pictures or videos. When it is a video, it can be compared using video frames at corresponding times. For example, a video can be captured by an image acquisition device as a safe in-vehicle image. Then, the image acquisition device parameter data that changes over time during the safe in-vehicle image acquisition can be obtained. Then, the current image acquisition device can be adjusted according to the image acquisition device parameter data that changes over time to obtain a video as a current in-vehicle image. During comparison, the starting time of the video can be taken as the starting point, and the video frames at each time the image acquisition device parameters change can be used as the comparison video frames. This can save the number of image acquisition devices and reduce the hardware cost of implementing this method. If the time of change of the image acquisition device parameters is t1, t2, t3, then a video frame before t1, a video frame between t1 and t2, a video frame between t2 and t3, and a video frame after t3 are selected. Both videos are selected according to this standard and compared accordingly by time. At this time, video frames captured by the same image acquisition device can be compared accordingly. It should be noted that the number of image acquisition devices can be one or more.

[0053] In another embodiment, acquiring a safe interior image of the vehicle to be tested includes, under safe conditions, acquiring images of multiple areas to be tested inside the vehicle using at least one image acquisition device installed on the vehicle to be tested, thereby obtaining multiple safe interior images; acquiring a current interior image of the vehicle to be tested includes, acquiring images of the areas to be tested inside the vehicle using at least one image acquisition device, thereby obtaining one or more current interior images, wherein the image acquisition device parameters of the image acquisition device are consistent with the image acquisition device parameters when acquiring the corresponding safe interior images; comparing the current interior image acquired by the same image acquisition device with the safe interior image, and if the current interior image is different from the safe interior image, triggering the step of inputting the current interior image into an image processing model.

[0054] As an example, the safety condition inside the vehicle can be the state it was in when it rolled off the production line, before any modifications were made. Alternatively, it can be a state deemed safe by qualified personnel. By re-evaluating the safety condition after the vehicle has rolled off the production line, users can make safe modifications to the vehicle, avoiding repeated safety risk checks on the same vehicle layout and further saving computing power.

[0055] Following the above embodiments, if the current in-vehicle image is the same as the safe in-vehicle image, then the in-vehicle safety risk of the area to be detected corresponding to the current in-vehicle image is not further assessed. Here, "not further assessing the in-vehicle safety risk of the area to be detected corresponding to the current in-vehicle image" can be understood as no longer inputting this current in-vehicle image into the image processing model and not executing the subsequent steps in this embodiment. Those skilled in the art can still use it for other analyses or as a basis for assessing in-vehicle safety risks in other dimensions. Of course, if computational waste is not considered, those skilled in the art can also continue to input these current in-vehicle images into the image processing model and execute the subsequent steps in this embodiment.

[0056] As an example, the image correspondence in Table 2 above can be used for one-to-one comparison to reduce computational power consumption. For identical images, no further operations are performed; only the current in-vehicle image with different comparison results is further processed for subsequent inference. This can save unnecessary computational power, improve detection speed, and to some extent, enhance the user experience.

[0057] In one embodiment, to further conserve computing power, if the current in-vehicle image differs from the safe in-vehicle image, the method further includes: determining the difference image region between the current in-vehicle image and the safe in-vehicle image; inputting the image of the difference image region as a difference image into the image processing model; and executing subsequent processes. The size of the difference image region can be set by those skilled in the art, but in principle, it should not be too small, making it impossible to distinguish the location of the difference image region in the cockpit, which is not conducive to the accurate acquisition of the current natural language description.

[0058] As an example, the comparison between the current in-vehicle image and the safe in-vehicle image can use a pre-trained model to calculate the feature vectors of the two images, and then calculate the similarity to determine whether the images have changed. This largely avoids the workload of training proprietary models, improves work efficiency, and saves costs. The comparison between the current in-vehicle image and the safe in-vehicle image can also employ other methods known to those skilled in the art, which will not be elaborated here.

[0059] When there are multiple safe in-vehicle images and multiple current in-vehicle images, by comparing the current in-vehicle images with the safe in-vehicle images, those current in-vehicle images that are identical to the safe in-vehicle images can be excluded. This indicates that the area to be detected corresponding to these images has not changed and there has been no modification or placement of other items. Therefore, they can not be used as reference images for subsequent safety risk detection, which can effectively save computing power.

[0060] The solution provided in the above embodiments employs a two-step method: the first step filters images that have changed, and the second step detects the changes in these images, reducing the actual computational load. That is, by detecting which parts of the image have changed, only the changed parts are analyzed in subsequent processing, saving subsequent computational load and time. If the two steps differ, it indicates that something inside the vehicle has been added, removed, or changed, causing the image to change.

[0061] Please see Figure 3 , Figure 3 A schematic flowchart illustrating the comparison between a current in-vehicle image and a safe in-vehicle image provided in an embodiment of this application is shown below. Figure 3 As shown, taking the image inside the safe vehicle as the original image and the current image inside the vehicle as the current image, the corresponding original and current images are input into a pre-trained deep learning model (such as ResNet, VGG, Inception, etc.) to extract feature vectors. Then, a similarity method is used to calculate whether the two vectors are similar. Change detection is performed based on the similarity. If the similarity is less than a preset similarity threshold, it indicates that the image has changed and requires further processing. Images with changes are detected and further processed. If no changes have occurred, i.e., the similarity is greater than or equal to the preset similarity threshold, the two images are considered identical and no processing is performed. The feature vector extraction method can be implemented using relevant models known to those skilled in the art, and will not be elaborated here.

[0062] As an example, image processing models include, but are not limited to, existing models such as the Palm-E model and the CLIP model. These models can generate corresponding natural language descriptions based on images. For instance, given an input image of a lighter on a dashboard, the model would output the following natural language description: "The current point of change is the dashboard. On the dashboard of a car, there is a lighter. The lighter is located on the right edge of the dashboard and is quite conspicuous. The lighter is made of plastic, with a smooth surface and a black color. The outline of the lighter is clear, the lines are simple, and there is no excessive decoration." Image processing models can utilize pre-trained models from related technologies, or they can be obtained by those skilled in the art through specific training of models from related technologies as needed.

[0063] Step S230: Generate a description statement for the current in-vehicle safety risk detection based on the current natural language description, the current in-vehicle environment information, and preset model prompts.

[0064] The current method for generating in-vehicle safety risk detection descriptions can be achieved by simply concatenating the current natural language description, current in-vehicle environment information, and preset model prompts. Alternatively, these three parts can be filled into the corresponding positions within a text template. Other text concatenation methods known to those skilled in the art can also be used to achieve this. Multiple text templates can be set, and selection can be based on at least one dimension, such as the vehicle's external environment, the current in-vehicle environment, or user preferences.

[0065] In another embodiment, the generation of the current in-vehicle safety risk detection description can also be combined with information about the vehicle's external environment, such as whether the vehicle is parked in an open or indoor location, whether the vehicle is being transported (e.g., en route to its destination by sea) or in normal driving mode. As an example, the vehicle's external environment information can be used as a separate component in the generation of the current in-vehicle safety risk detection description, or it can be integrated into preset model prompts for subsequent generation of the current in-vehicle safety risk detection description.

[0066] As an example, the system integrates three parts of information: preset model prompts, in-vehicle environment information, and image description (current natural language description). The integrated input is similar to: "You are a driver with 20 years of experience and have extensive experience in identifying in-vehicle safety hazards. Please determine whether there are any safety hazards in the passenger compartment based on the following description. If there are, please briefly describe them: The current in-vehicle temperature is 25 degrees Celsius, the in-vehicle humidity is 45%RH, the in-vehicle PM2.5 concentration is 35 micrograms per cubic meter, and the in-vehicle PM10 concentration is 50 micrograms per cubic meter. The current point of change is the dashboard. On the dashboard, there is a lighter. The lighter is located on the right edge of the dashboard and is quite conspicuous. The lighter is made of plastic with a smooth surface and is black in color. The lighter has a clear outline, simple lines, and no excessive decoration."

[0067] The above steps can generate text for reasoning by a large language model, which is simpler and faster than simple image recognition, and is not limited by a database of dangerous items, making detection more flexible.

[0068] Step S240: Input the current in-vehicle safety risk detection description statement into the inference language model to make a prediction on whether there is an in-vehicle safety risk in the vehicle to be detected, and obtain the in-vehicle safety risk detection result of the vehicle to be detected.

[0069] The reasoning language model itself possesses a certain degree of reasoning ability, enabling it to make relatively reasonable inferences based on information provided by the user. By utilizing the reasoning capabilities of the reasoning language model to identify in-vehicle safety hazards, the coverage of safety identification can be effectively improved, demonstrating a certain degree of recognition for various safety risks. Based on the input description of the current in-vehicle safety risk detection, the reasoning language model returns a corresponding response. If a positive response is received, meaning the in-vehicle safety risk detection result is dangerous, then there is a safety hazard inside the vehicle. If a negative response is received, meaning the in-vehicle safety risk detection result is safe, then there is no safety hazard inside the vehicle. Once a positive response is received, the information needs to be relayed to the vehicle owner.

[0070] The reasoning language model can be a model that has been trained in related technologies, or it can be a model obtained by those skilled in the art through further adaptive training of the model in related technologies. For example, the model can be trained using the corresponding sample in-vehicle safety risk detection description statements and the corresponding sample detection results to obtain a reasoning language model that can be applied to this embodiment.

[0071] As mentioned in the foregoing embodiments, the output of the inference big language model includes at least the in-vehicle safety risk detection result, which may represent either danger or safety. As an example, the output may also include inference analysis content, which represents the reasons and basis for making the in-vehicle safety risk detection result, as well as other speculative content output by the inference big language model.

[0072] In one embodiment, after obtaining the in-vehicle safety risk detection result of the vehicle to be detected, the method further includes: acquiring a new current in-vehicle image of the vehicle to be detected; if the new current in-vehicle image is different from the safe in-vehicle image, comparing the new current in-vehicle image with all stored target historical in-vehicle images, wherein the target historical in-vehicle images are images acquired by the same image acquisition device as the new current in-vehicle image under the same image acquisition device parameters, and the target historical in-vehicle images are current in-vehicle images historically input into the image processing model, and the in-vehicle environment information corresponding to the target historical in-vehicle images is the same as or similar to the new in-vehicle environment information corresponding to the new current in-vehicle image; if the new current in-vehicle image is different from all stored target historical in-vehicle images, triggering the step of inputting the new current in-vehicle image into the image processing model; if the new current in-vehicle image is the same as a stored target historical in-vehicle image, using the in-vehicle safety risk detection sub-result corresponding to the target historical in-vehicle image as the in-vehicle safety risk detection sub-result of the new current in-vehicle image, wherein the in-vehicle safety risk detection result includes the in-vehicle safety risk detection sub-result output by the inference big language model based on the current natural language description of each current in-vehicle image.

[0073] It is understandable that the increase or decrease of one or more items inside a vehicle may be a long-term state. Under the premise of the same or similar in-vehicle environment, or in other words, the parameter changes in the in-vehicle environment information are within a preset range of change, it can be considered that small changes in the in-vehicle environment have little impact on risk detection. Therefore, it is unnecessary to consider changes in detection results due to changes in in-vehicle environment information. Since the change in the item has already been detected, detecting the same change again would waste computational resources. To address this, after obtaining a new current in-vehicle image, it is first compared with a safe in-vehicle image. If they are different, it is then compared with previously detected target historical in-vehicle images. If they are still different, the new current in-vehicle image is used as the current in-vehicle image in step S110 for the next step. If the new current in-vehicle image is the same as a previous current in-vehicle image M input to the image processing model, then the prediction result of that current in-vehicle image M can be directly used without repeatedly calling the model for prediction.

[0074] Considering that the items inside the vehicle may not change frequently, storing and recording images of the vehicle interior that have been verified as safe, and then comparing new images of the changed vehicle interior with the previously safe images, can improve detection efficiency to some extent.

[0075] In one embodiment, after obtaining the in-vehicle safety risk detection result of the vehicle to be detected, the method further includes: if the in-vehicle safety risk detection result is dangerous, generating alarm information based on the current in-vehicle image of the input image processing model and the reasoning analysis content output by the reasoning big language model; and feeding back the alarm information to a preset prompt terminal through the in-vehicle communication device.

[0076] The preset prompt terminal can be a terminal pre-set by those skilled in the art, such as the car owner's mobile phone or the computer of an authorized party of the car owner.

[0077] The system also sends the changing images of the vehicle interior to the preset notification terminal, which helps users to intuitively understand which items are causing the risks inside the vehicle and to determine whether action is needed.

[0078] Following the above embodiments, after the alarm information is fed back to the preset prompt terminal via the in-vehicle communication device, the method further includes: obtaining a manual confirmation instruction; in response to the manual confirmation instruction, adjusting at least some of the image acquisition devices to capture real-time images of the vehicle to be detected; and feeding back the real-time images to the preset prompt terminal via the in-vehicle communication device.

[0079] It is understandable that after receiving an alarm message, the preset notification terminal can display the alarm information to a specific user. If the user needs it, they can send a manual confirmation command to the vehicle to be inspected. This command includes adjusting the parameters of the image acquisition device so that it can be adjusted to the angle required by the user, and performing zooming and other operations according to the user's requirements to display the latest image of the vehicle's interior. It should be noted that the real-time image here can be an image transmitted via real-time transmission technology, or an image acquired in response to a manual confirmation command. The image is not necessarily limited to "real-time"; in other words, the real-time image can be real-time or a delayed image with a reasonable time lag. By feeding these real-time images back to the preset notification terminal, it is easier for the user to view and perform further processing.

[0080] The in-vehicle safety risk detection method provided in the above embodiments acquires preset model prompts, as well as the current in-vehicle image, the safe in-vehicle image, and the current in-vehicle environment information of the vehicle to be detected. If the current in-vehicle image differs from the safe in-vehicle image, the current in-vehicle image is input into an image processing model to obtain a current natural language description of the current in-vehicle image. Based on the current natural language description, the current in-vehicle environment information, and the preset model prompts, a current in-vehicle safety risk detection description statement is generated. This description statement is then input into a large-scale inference language model to infer whether the vehicle to be detected has an in-vehicle safety risk, thus obtaining the detection risk. The vehicle interior safety risk detection results enable the identification of potential safety hazards inside the vehicle, improving passenger compartment safety. Leveraging the extensive knowledge reasoning capabilities of the inference language model, the system effectively enhances safety identification coverage by utilizing its reasoning abilities to identify in-vehicle safety hazards. It has a certain degree of identifiability for various safety hazards. Combining in-vehicle environmental information with the item information provided by the current natural language description to comprehensively determine the existence of safety hazards further increases the accuracy of in-vehicle safety risk assessment. This method is no longer affected by the comprehensiveness of a pre-established hazardous materials database, improving detection reliability and effectively enhancing driving safety.

[0081] The following specific embodiment illustrates the in-vehicle safety risk detection method provided in the above embodiments. Please refer to [link / reference]. Figure 4 , Figure 4 A schematic flowchart illustrating a specific method for detecting in-vehicle safety risks provided in an embodiment of this application is shown below. Figure 4As shown, the process begins with raw image acquisition, obtaining the initial image information of the passenger compartment. This image is the unaltered initial image of the passenger compartment, i.e., the aforementioned safe interior image. Next, the current image is acquired. After the owner purchases the vehicle, the passenger compartment may have undergone modifications or changes due to aging. In this case, the current image information of the passenger compartment needs to be acquired, i.e., the current interior image. Then, change detection is performed, checking whether the current interior image has changed compared to the safe interior image (whether the two images are identical). If a change has occurred, it is input into the image detection model; otherwise, the process ends. Next, the model judgment part is executed. Combining the interior environment information, the current natural language description of the changed image, and preset model prompts, a large language model is used to determine whether a safety hazard exists. Finally, the alarm notification part proactively notifies the owner if a safety hazard exists in the passenger compartment. If no safety hazard exists, the process ends.

[0082] Compared to in-vehicle safety risk detection solutions in related technologies, the in-vehicle safety risk detection method provided in this embodiment establishes a vehicle factory baseline image library (generated based on acquired safe in-vehicle images). It uses in-vehicle cameras to collect initial images from various directions inside the vehicle, preparing for subsequent change detection. By comparing subsequent data (the current in-vehicle image) with factory data (the safe in-vehicle image corresponding to the current in-vehicle image), it determines whether any safety hazards exist. Unlike previous monitoring methods that primarily focused on monitoring external anomalies, this method can investigate in-vehicle safety hazards, improving passenger compartment safety. In change detection, a pre-trained model is used to calculate the feature vectors of two images, and then the similarity is calculated to determine whether the images have changed. The pre-trained image feature extraction model can use pre-trained models from related technologies, largely avoiding the workload of training proprietary models, improving efficiency, and saving costs. This method employs a two-step approach: the first step filters for images showing changes, and the second step detects these changes (by inputting different current in-vehicle images into the image processing model), reducing the actual computational load. The current in-vehicle image is converted into a natural language description using an image processing model. The reasoning capabilities of a large-scale inference language model are then used to identify potential safety hazards in the passenger compartment. This large-scale inference language model possesses extensive knowledge reasoning capabilities. Using its reasoning ability to identify in-vehicle safety hazards effectively improves safety identification coverage and provides a certain degree of recognition for various safety hazards. Furthermore, by combining in-vehicle environmental information and object information (the current natural language description) to comprehensively determine the existence of safety hazards, and considering that the same object may have different safety impacts under different in-vehicle environmental conditions, the results of in-vehicle risk detection using this method are more accurate and reliable.

[0083] In one embodiment, an in-vehicle safety risk detection device is provided, which is used to perform the in-vehicle safety risk detection method provided in any of the above embodiments. Please refer to [link to previous document]. Figure 5 , Figure 5 A schematic diagram of the in-vehicle safety risk detection device provided in an embodiment of this application is shown below. Figure 5 As shown, the in-vehicle safety risk detection device 500 includes an acquisition module 510, an image description module 520, a statement generation module 530, and a detection module 540. The acquisition module 510 acquires preset model prompts, as well as the current in-vehicle image, the safe in-vehicle image, and the current in-vehicle environment information of the vehicle to be detected. The image description module 520, if the current in-vehicle image differs from the safe in-vehicle image, inputs the current in-vehicle image into an image processing model to obtain a current natural language description of the current in-vehicle image. The statement generation module 530 generates a current in-vehicle safety risk detection description statement based on the current natural language description, the current in-vehicle environment information, and the preset model prompts. The detection module 540 inputs the current in-vehicle safety risk detection description statement into a large-scale inference language model to infer whether the vehicle to be detected has in-vehicle safety risks, and obtains the in-vehicle safety risk detection result for the vehicle to be detected.

[0084] In one embodiment, the acquisition module is configured to: acquire the image acquisition device parameters of the safe interior image of the vehicle to be detected before acquiring the current interior image of the vehicle to be detected; adjust the parameters of the image acquisition device of the vehicle to be detected based on the image acquisition device parameters; and acquire the current interior image through the adjusted image acquisition device.

[0085] In one embodiment, see further. Figure 5 The device also includes a comparison module 550, and the acquisition module is further configured to: acquire safe interior images of the vehicle to be tested, including, under safe conditions inside the vehicle, acquiring images of multiple areas to be tested inside the vehicle using at least one image acquisition device installed in the vehicle to be tested, to obtain multiple safe interior images; acquiring images of the areas to be tested inside the vehicle using at least one image acquisition device, to obtain one or more current interior images, wherein the image acquisition device parameters of the image acquisition device are consistent with the image acquisition device parameters when acquiring the corresponding safe interior images; the comparison module is used to compare the current interior images acquired by the same image acquisition device with the safe interior images, and if the current interior images are different from the safe interior images, triggering the statement generation module to input the current interior images into the image processing model.

[0086] Following the above embodiments, the statement generation module is further configured to: if the current in-vehicle image is the same as the safe in-vehicle image, then no further assessment of the in-vehicle safety risk of the area to be detected corresponding to the current in-vehicle image is performed.

[0087] In one embodiment, the comparison module is further configured to acquire the current in-vehicle image and the safe in-vehicle image of the vehicle to be detected, and then match the current in-vehicle image and the safe in-vehicle image based on the device identifier and image acquisition device parameters of the image acquisition device at the time of image acquisition to obtain one or more image pairs; input the current in-vehicle image and the safe in-vehicle image into a preset image feature extraction model to obtain the current image feature vector and the safe image feature vector; calculate the similarity between the current image feature vector and the safe image feature vector of the same image pair. If the similarity is less than a preset similarity threshold (a specific threshold can be set by those skilled in the art as needed), then it is determined that the current in-vehicle image and the safe in-vehicle image of the same image pair are different.

[0088] In one embodiment, see further. Figure 5 The device also includes an alarm module 560, which, after obtaining the in-vehicle safety risk detection result of the vehicle to be detected, generates alarm information based on the current in-vehicle image of the input image processing model and the reasoning analysis content output by the reasoning big language model if the in-vehicle safety risk detection result is dangerous; and feeds back the alarm information to the preset prompt terminal through the in-vehicle communication device.

[0089] Following the above embodiments, the alarm module is further configured to: obtain a manual confirmation instruction; in response to the manual confirmation instruction, adjust at least some of the image acquisition devices to capture real-time images of the vehicle to be detected; and feed back the real-time images to a preset prompt terminal via an in-vehicle communication device.

[0090] Specific limitations regarding in-vehicle safety risk detection devices can be found in the limitations of in-vehicle safety risk detection methods described above, and will not be repeated here. Each module in the aforementioned in-vehicle safety risk detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the electronic device, or stored in the memory of the electronic device as software, so that the processor can call and execute the corresponding operations of each module.

[0091] In this embodiment, the in-vehicle safety risk detection device is essentially equipped with multiple modules to execute the in-vehicle safety risk detection method in any of the above embodiments. The specific functions and technical effects can be referred to in the above embodiments, and will not be repeated here.

[0092] In one embodiment, a vehicle is provided, which includes the in-vehicle safety risk detection device provided in any of the above embodiments. The specific functions and technical effects of the vehicle can be referred to the above embodiments, and will not be repeated here.

[0093] In one embodiment, a vehicle is provided, comprising an image acquisition device, an information acquisition module, an image comparison module, an image description module, a statement generation module, a sending module, and a receiving module, wherein: the image acquisition device is used to acquire current in-vehicle images and safe in-vehicle images of the vehicle to be detected; the information acquisition module is used to acquire preset model prompts and current in-vehicle environment information; the image comparison module is used to compare the current in-vehicle image with the safe in-vehicle image; the image description module is used to input the current in-vehicle image into an image processing model to obtain a current natural language description of the current in-vehicle image if the current in-vehicle image is different from the safe in-vehicle image; the statement generation module is used to generate a current in-vehicle safety risk detection description statement based on the current natural language description, the current in-vehicle environment information, and the preset model prompts; the sending module is used to send the current in-vehicle safety risk detection description statement to an inference large language model, and use the inference large language model to infer whether there is an in-vehicle safety risk in the vehicle to be detected, thereby obtaining the in-vehicle safety risk detection result of the vehicle to be detected; and the receiving module is used to receive the in-vehicle safety risk detection result.

[0094] For specific limitations regarding the vehicle, please refer to the limitations on in-vehicle safety risk detection methods mentioned above, which will not be repeated here. The various modules in the aforementioned vehicle can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in the electronic device, or stored in software within the memory of the electronic device, so that the processor can call and execute the corresponding operations of each module.

[0095] In this embodiment, the vehicle is actually equipped with multiple modules to execute the in-vehicle safety risk detection method in any of the above embodiments. The specific functions and technical effects can be referred to in the above embodiments, and will not be repeated here.

[0096] See Figure 6 , Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown below. Figure 6 As shown, this embodiment of the invention also provides an electronic device 600, including a processor 601, a memory 602, and a communication bus 603; the communication bus 603 is used to connect the processor 601 and the memory 602; the processor 601 is used to execute a computer program stored in the memory 602 to implement the method described in any of the above embodiments.

[0097] This invention also provides a computer-readable storage medium having a computer program stored thereon, the computer program being used to cause a computer to perform the method described in any of the above embodiments.

[0098] This application also provides a non-volatile readable storage medium storing one or more modules (programs) that, when applied to a device, enable the device to execute the instructions included in the steps provided in this application.

[0099] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0100] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0101] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0102] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0104] It should be understood that the terms "first," "second," etc., used in this application are used to distinguish similar objects and do not necessarily indicate a specific order or sequence. The technical features to which these terms are used can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0105] It should be understood that although the flowcharts provided in the embodiments of this application indicate the various steps with arrows, the order indicated by the arrows does not necessarily limit the implementation order of these steps. Those skilled in the art can perform these steps in other orders according to different implementation scenarios and requirements.

[0106] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for detecting in-vehicle safety risks, characterized in that, The method includes: Obtain preset model prompts, as well as the current in-vehicle image, safe in-vehicle image, and current in-vehicle environment information of the vehicle to be detected; If the current in-vehicle image is different from the safe in-vehicle image, the current in-vehicle image is input into the image processing model to obtain the current natural language description of the current in-vehicle image; Generate a description of the current in-vehicle safety risk detection based on the current natural language description, the current in-vehicle environment information, and the preset model prompt words; The current in-vehicle safety risk detection description is input into the inference language model to infer whether the vehicle under test has in-vehicle safety risks, and the in-vehicle safety risk detection result of the vehicle under test is obtained.

2. The in-vehicle safety risk detection method as described in claim 1, characterized in that, Before acquiring the current interior image of the vehicle to be detected, the method further includes: Parameters of the image acquisition device used to obtain safe in-vehicle images of the vehicle to be detected; The parameters of the image acquisition device for the vehicle to be detected are adjusted based on the parameters of the image acquisition device. The current in-vehicle image is acquired using the adjusted image acquisition device.

3. The in-vehicle safety risk detection method as described in claim 1, characterized in that, Acquiring safe interior images of the vehicle to be inspected includes, under safe conditions inside the vehicle, acquiring images of multiple areas to be inspected inside the vehicle using at least one image acquisition device installed in the vehicle to be inspected, thereby obtaining multiple safe interior images. Acquiring the current in-vehicle image of the vehicle to be detected includes acquiring images of the area to be detected inside the vehicle using at least one image acquisition device to obtain one or more current in-vehicle images. The image acquisition device parameters of the image acquisition device are consistent with the image acquisition device parameters when acquiring the corresponding safe in-vehicle image. The current in-vehicle image captured by the same image acquisition device is compared with the safe in-vehicle image. If the current in-vehicle image is different from the safe in-vehicle image, the step of inputting the current in-vehicle image into the image processing model is triggered. If the current in-vehicle image is the same as the safe in-vehicle image, then no further assessment of the in-vehicle safety risk of the area to be detected corresponding to the current in-vehicle image will be conducted.

4. The in-vehicle safety risk detection method as described in claim 3, characterized in that, The current in-vehicle environment information includes at least one of the following: in-vehicle temperature, in-vehicle humidity, in-vehicle fine particulate matter concentration, and in-vehicle inhalable particulate matter concentration.

5. The in-vehicle safety risk detection method according to any one of claims 1-4, characterized in that, After acquiring the current in-vehicle image and the safe in-vehicle image of the vehicle to be detected, the method further includes: Based on the device identifier and parameters of the image acquisition device at the time of image acquisition, the current in-vehicle image and the safe in-vehicle image are matched to obtain one or more image pairs; The current in-vehicle image and the safe in-vehicle image are input into a preset image feature extraction model to obtain the current image feature vector and the safe image feature vector. Calculate the similarity between the current image feature vector and the safe image feature vector for the same image pair; If the similarity is less than a preset similarity threshold, then the current in-vehicle image of the same image pair is determined to be different from the safe in-vehicle image.

6. The in-vehicle safety risk detection method according to any one of claims 1-4, characterized in that, After obtaining the in-vehicle safety risk detection results of the vehicle to be tested, the method further includes: If the in-vehicle safety risk detection result is dangerous, an alarm message is generated based on the current in-vehicle image of the input image processing model and the reasoning analysis content output by the reasoning big language model; The alarm information is fed back to a preset notification terminal via the in-vehicle communication device.

7. The in-vehicle safety risk detection method as described in claim 6, characterized in that, After transmitting the alarm information to a preset notification terminal via the in-vehicle communication device, the method further includes: Receive manual confirmation instruction; In response to the manual confirmation command, at least some of the image acquisition devices are adjusted to capture real-time images of the vehicle to be detected; The real-time images are fed back to a preset notification terminal via the in-vehicle communication device.

8. A vehicle interior safety risk detection device, characterized in that, The device includes: The acquisition module is used to acquire preset model prompt words, as well as the current in-vehicle image, safe in-vehicle image, and current in-vehicle environment information of the vehicle to be detected; The image description module is used to input the current in-vehicle image into the image processing model to obtain a current natural language description of the current in-vehicle image if the current in-vehicle image is different from the safe in-vehicle image. The statement generation module is used to generate a current in-vehicle safety risk detection description statement based on the current natural language description, the current in-vehicle environment information, and preset model prompt words; The detection module is used to input the current in-vehicle safety risk detection description statement into the inference language model to speculate whether the vehicle under test has in-vehicle safety risks, and to obtain the in-vehicle safety risk detection result of the vehicle under test.

9. A vehicle, characterized in that, The vehicle includes an image acquisition device, an information acquisition module, an image comparison module, an image description module, a statement generation module, a sending module, and a receiving module, wherein: The image acquisition device is used to acquire current in-vehicle images and safe in-vehicle images of the vehicle to be detected; The information acquisition module is used to acquire preset model prompt words and current in-vehicle environment information; The image comparison module is used to compare the current in-vehicle image with a safe in-vehicle image; The image description module is used to input the current in-vehicle image into the image processing model to obtain a current natural language description of the current in-vehicle image if the current in-vehicle image is different from the safe in-vehicle image. The statement generation module is used to generate a current in-vehicle safety risk detection description statement based on the current natural language description, the current in-vehicle environment information, and preset model prompt words; The sending module is used to send the current in-vehicle safety risk detection description statement to the inference big language model, and use the inference big language model to infer whether the vehicle to be detected has in-vehicle safety risks, and obtain the in-vehicle safety risk detection result of the vehicle to be detected. The receiving module is used to receive the in-vehicle safety risk detection results.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images