Vehicle control method and device based on large model, equipment and storage medium

Abstract

The present application relates to a kind of vehicle control method based on large model, device, equipment and storage medium, comprising: by the information acquisition equipment installed on vehicle, the state information of object in vehicle in preset time period and the current environmental information in vehicle are determined;Wherein, state information represents the performance state of object in vehicle;According to the state information of object in vehicle in preset time period and the current environmental information in vehicle, based on the large model of pre-set, the initial demand information of object in vehicle is predicted, and initial demand information is fed back to user;Wherein, initial demand information represents the control scheme that object in vehicle needs to be executed by in-vehicle control device predicted;In response to the feedback instruction of the initial demand information of user, target demand information is determined, and according to target demand information, corresponding in-vehicle control device is controlled.Combining the demand of large model prediction object in vehicle, thereby accurately controlling vehicle, improve the ride experience of object in vehicle.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and specifically to a vehicle control method, apparatus, device, and storage medium based on a large model. Background Technology

[0002] As people's living standards improve, their demands for vehicle control are also increasing. In particular, with the growing proportion of pet owners, how to travel with pets by car has become a major issue for pet owners.

[0003] Prolonged coexistence of people and pets in the relatively enclosed environment of a car poses a potential threat to the health of both passengers and pets. Furthermore, drivers cannot monitor their pets' condition while driving, thus hindering vehicle control efficiency and precision, and ultimately reducing the passenger experience and safety. Summary of the Invention

[0004] The purpose of this invention is to provide a vehicle control method, device, equipment, and storage medium based on a large model, so as to improve the control efficiency and accuracy of vehicles and enhance the riding experience.

[0005] In a first aspect, the present invention provides a vehicle control method based on a large model, comprising:

[0006] By using information collection devices installed on the vehicle, the status information of objects inside the vehicle and the current environmental information inside the vehicle are determined within a preset time period; wherein, the status information represents the performance status of the objects inside the vehicle.

[0007] Based on the state information of the in-vehicle object within a preset time period and the current environmental information inside the vehicle, the initial demand information of the in-vehicle object is predicted based on a preset large model, and the initial demand information is fed back to the user; wherein, the initial demand information represents the control scheme that the predicted in-vehicle object needs to be executed by the in-vehicle control device.

[0008] In response to the user's feedback instruction on the initial demand information, the target demand information is determined, and the corresponding in-vehicle control equipment is controlled according to the target demand information.

[0009] Secondly, the present invention provides a vehicle control device based on a large model, comprising:

[0010] The information determination module is used to determine the status information of objects inside the vehicle and the current environmental information inside the vehicle within a preset time period through information collection devices installed on the vehicle; wherein, the status information represents the performance status of the objects inside the vehicle.

[0011] The demand prediction module is used to predict the initial demand information of the in-vehicle objects based on the state information of the objects in the vehicle within a preset time period and the current environmental information inside the vehicle, and to feed back the initial demand information to the user; wherein, the initial demand information represents the control scheme that the predicted in-vehicle objects need to be executed by the in-vehicle control equipment.

[0012] The vehicle control module is used to respond to the user's feedback command on the initial demand information, determine the target demand information, and control the corresponding in-vehicle control equipment according to the target demand information.

[0013] Thirdly, the present invention provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0014] The memory stores computer-executed instructions;

[0015] The processor executes computer execution instructions stored in the memory to implement the method as described in the first aspect.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.

[0017] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0018] This invention provides a vehicle control method, device, equipment, and storage medium based on a large model. Various information acquisition devices are installed on the vehicle to obtain the state information of objects inside the vehicle and the current environmental information within the vehicle. A large model is pre-constructed. Based on the state information of objects inside the vehicle over a period of time and the current environmental information, the pre-built large model can predict what operations the objects inside the vehicle might expect from the in-vehicle control devices. The predicted results are used as the initial demand information of the objects inside the vehicle and fed back to the user. The user provides feedback on the initial demand information, and the system responds to the user's feedback command to determine the target demand information. Based on the target demand information, the corresponding in-vehicle control devices are controlled to perform the appropriate operations. This enables timely prediction of the needs of objects inside the vehicle, allowing the in-vehicle control devices to make targeted adjustments, avoiding user distraction while driving, and improving vehicle control accuracy and efficiency. Through human-computer interaction with the user, the system can meet the user's actual needs and enhance the driving experience. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0020] Figure 1 A flowchart illustrating a vehicle control method based on a large model, provided as an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the pet mode interface provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram illustrating the processing steps of different pet modes provided in embodiments of the present invention;

[0023] Figure 4 This is a diagram illustrating the operation of an in-vehicle control device provided in an embodiment of the present invention.

[0024] Figure 5 A flowchart illustrating a vehicle control method based on a large model, provided as an embodiment of the present invention;

[0025] Figure 6 A flowchart illustrating the workflow of a large model provided in this embodiment of the invention;

[0026] Figure 7 A structural block diagram of a vehicle control device based on a large model provided in an embodiment of the present invention;

[0027] Figure 8 A structural block diagram of a vehicle control device based on a large model provided in an embodiment of the present invention;

[0028] Figure 9 A structural block diagram of an electronic device provided in an embodiment of the present invention;

[0029] Figure 10 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.

[0030] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0031] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0032] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0033] In the description of this invention, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. Furthermore, in the description of this invention, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0034] It should be noted that, due to space limitations, this specification does not exhaustively list all possible implementation methods. Those skilled in the art, after reading this specification, should be able to deduce that any combination of technical features can constitute an optional implementation method, provided that the technical features do not contradict each other. The following provides a detailed description of each embodiment.

[0035] As people's living standards continue to improve, driving has gradually become the mainstream mode of transportation, and users often need to travel with pets or infants while driving. However, users cannot always keep an eye on their pets or infants while driving, and cannot promptly operate the vehicle's control devices, affecting the driving experience and safety.

[0036] With the increasing proportion of pet owners, traveling with pets has become a major challenge. For example, during long journeys, the relatively enclosed space of a car can become stuffy and prone to bacterial growth, posing a potential threat to the health of both passengers and pets. Furthermore, when pets experience anxiety or hunger while driving, owners are often unable to attend to them. When owners temporarily leave the vehicle, pets left unattended pose a risk of suffocation, especially in hot weather. To address these issues, real-time monitoring of pets and precise vehicle control based on monitoring results are necessary to soothe pets, reduce driver stress, and improve driving safety.

[0037] The present invention provides a vehicle control method, device, equipment and storage medium based on a large model, which aims to solve the above-mentioned technical problems of the prior art.

[0038] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0039] Figure 1 This is a flowchart illustrating a vehicle control method based on a large model according to an embodiment of the present invention. This method can be executed by a vehicle control device based on a large model. Figure 1 As shown, the method includes the following steps:

[0040] S101. Using information collection equipment installed on the vehicle, determine the status information of objects inside the vehicle and the current environmental information inside the vehicle within a preset time period; wherein, the status information represents the performance status of objects inside the vehicle.

[0041] For example, various information collection devices can be installed on the vehicle, such as cameras and microphones. These devices can be used to collect various information inside and outside the vehicle in real time or at set intervals. Objects inside the vehicle can be all living objects except the driver, such as infants or pets. The information collection devices can acquire the status information of each object inside the vehicle in real time. This status information represents the behavior of the objects inside the vehicle; for example, it can be data such as images, videos, and audio recordings of the objects. The information collection devices can be installed in various locations within the vehicle to obtain status information of objects at different locations. For example, if a pet is located in the back seat, video data of the pet can be collected using a camera facing the back seat to determine if the pet's current actions have changed significantly, if there are any physical abnormalities, or if it is barking. A time period can be preset to acquire the status information of objects inside the vehicle within that preset time period; for example, status information for 10 minutes can be acquired.

[0042] In-vehicle information collection devices can also include various sensors such as sound sensors, temperature sensors, humidity sensors, and odor sensors. These devices can collect real-time or periodic environmental information within the vehicle. For example, in-vehicle environmental information can include temperature, humidity, and air quality. Taking an air quality sensor as an example, in scenarios where people and pets are present, the air quality sensor can detect the concentration of harmful gases in the air inside the vehicle, thereby determining whether the air inside the vehicle is polluted.

[0043] Vehicles can use cameras, microphones, and other devices to monitor the presence of any objects inside the vehicle other than the driver in real time. If an object other than the driver is detected entering the vehicle, the information collection device can be automatically activated to obtain the object's status information and the current environment inside the vehicle. For example, if a pet is detected entering the vehicle, the information collection device can be activated. The information collection device can also be activated via user instructions, such as voice commands from the driver or button presses on the vehicle's infotainment system or mobile device. The mobile device can be a user's smartphone or other similar device.

[0044] In this embodiment, when a pet is detected getting into the vehicle, a pet mode page can be displayed on the vehicle's infotainment system or the user's mobile terminal. The pet mode can include a vehicle entry mode. Taking the pet entry mode as an example, when the in-vehicle camera accurately captures the moment a pet gets into the vehicle, the system will immediately and automatically switch to this mode. The user will see the vehicle entry mode page and can instruct the user to activate the information collection device. The page can be customized; for example, a pet avatar display area can be set on the right side of the page, allowing users to upload personalized photos of their pets. The uploaded pet photos can be intelligently processed based on a deep learning image style transfer algorithm, giving them a unique AI-generated artistic style. When the pet mode is successfully triggered, the pet avatar on the page will be presented with a vivid dynamic effect. Specifically, the avatar can first open slowly and gracefully on the page in an elegant rotating manner, then gradually shrink and precisely position itself in the display area on the right side of the page. The vehicle infotainment system can also be equipped with translation software. When a pet is detected getting into the vehicle, the pet category can be identified, and the translation software can translate pet language in real time.

[0045] In this embodiment, the method further includes: determining the object category of the object inside the vehicle; if the object category of the object inside the vehicle is a preset category, then when there is only an object inside the vehicle, the preset button inside the vehicle is adjusted to a shielded mode; wherein, the shielded mode indicates that the button will not be triggered by pressing operation.

[0046] Specifically, for each object inside the vehicle, its object category can be determined. The object category could be a pet, an infant, etc. For pets inside the vehicle, the object category could also be a specific pet type, such as a cat or a dog. An image of the object can be acquired when it gets into the vehicle, and its object category can be determined based on a preset image recognition algorithm. In this embodiment, the preset image recognition algorithm is not specifically limited.

[0047] One or more object categories are pre-defined as preset categories, for example, the preset category is "dog". If the object category of the object inside the car is determined to be a preset category, it can be determined in real time whether the object is left alone in the car. For example, the pet mode can include a "left-behind" mode, which means that the pet is left alone in the car. If it is detected that only the pet is in the car, the preset buttons in the vehicle can be switched to a "disabled" mode, which means that the buttons will not be triggered by pressing. The preset buttons in the car can include the central locking button, the interior window button, the rearview mirror button, etc. By setting the disabled mode, the accidental pressing of the buttons by the pet can be prevented, ensuring the pet's safety.

[0048] When an object is left alone inside the vehicle, the information collection device can determine the object's status and the current environment inside the vehicle, and then continue executing S102 and S103 in this embodiment. Since the user is not in the vehicle, the initial request information can be fed back to the user through the user's mobile terminal, and feedback instructions from the user can be received.

[0049] The advantage of this setup is that it can disable the central locking buttons, interior window buttons, and rearview mirror buttons in scenarios where only one person is left in the car, thus preventing accidental button presses and improving the safety of the person inside the car.

[0050] In this embodiment, the vehicle is equipped with a tracking robot, which is also equipped with an information collection device. The system further includes: if the object category of the object inside the vehicle is a preset category, a tracking request is sent to the user when the object leaves the vehicle; the tracking request is used to prompt the user to confirm whether to track the object inside the vehicle using the tracking robot; if the user confirms the tracking request, the system controls the tracking robot to follow the object inside the vehicle and acquires the status information of the object collected by the tracking robot; the status information is then fed back to the user.

[0051] Specifically, the vehicle can be equipped with a tracking robot. This tracking robot is an independent device that can communicate with the vehicle's infotainment system or the user's terminal device. The tracking robot can also be equipped with information collection devices such as cameras and microphones, as well as output devices such as speakers.

[0052] If the object inside the vehicle is a preset category, such as a pet, it can monitor in real time whether the pet gets out of the car alone. This pet-only exit mode can be designated as a tracking mode. That is, tracking mode occurs when the pet leaves the car, but the user remains inside. For example, if the vehicle is parked in front of a veterinary clinic and the pet enters alone, tracking may be necessary to prevent the pet from getting lost. If it is confirmed that the pet has left the vehicle alone, a tracking request can be sent to the user via the vehicle's infotainment system or the user's device. The tracking request prompts the user to confirm whether a tracking robot is needed to track the object inside the vehicle.

[0053] If the user issues a command to stop tracking, the tracking robot will remain in the vehicle; if the user issues a command to confirm tracking, the tracking robot will also leave the vehicle and follow the pet in real time. The tracking robot uses its onboard data collection devices to collect real-time video and other status information about the pet, and then feeds this information back to the vehicle's infotainment system or the user's device.

[0054] In other words, in scenarios where the person and pet are separated, with the pet getting out of the car and the person remaining inside, the user can use voice commands to wake up the tracking robot to get out of the car and follow the pet, transmitting its status information to the vehicle's infotainment system via a communication system. The vehicle's infotainment system then transmits the received status information to the in-car camera and microphone, allowing the user to check on the pet and interact with it. The tracking robot uses an integrated distance sensor system to track and follow the pet's movements in real time and accurately, ensuring a tracking error of ≤10cm. When pet tracking mode is activated, the tracking robot uses an integrated visual sensor to capture and transmit video images of the pet to the vehicle's infotainment system in real time, while simultaneously capturing and transmitting the pet's audio signals to the vehicle's infotainment system via an auditory sensor. When the in-car camera detects that the pet has entered the vehicle, the tracking mode can be exited, and an exit signal is transmitted to the tracking robot via the communication system, causing the robot to stop at a preset position in the car.

[0055] The advantage of this setup is that, in scenarios where objects leave the vehicle, the robot can track the objects and transmit their status back, improving the safety of the objects and thus enabling human-computer interaction in various scenarios, thereby enhancing the user experience.

[0056] Figure 2 This is a schematic diagram of the pet mode interface. Figure 2 The display can be accessed through the vehicle's infotainment system or a mobile device. The right side of the screen shows images of objects inside the vehicle, while the left side displays various pet modes: "Pet Gets in" mode, "Leave Alone" mode when the owner leaves the vehicle, and "Tracking" mode when the pet gets out of the vehicle alone. When the vehicle is in a pet mode, the corresponding control will be illuminated, and the user can also manually touch the control to switch pet modes.

[0057] Figure 3 This diagram illustrates the processing steps for different pet modes. Pet modes can include in-car mode, co-habitation mode, pet-only mode, and tracking mode. When a person and pet are both in the car, they can be in co-habitation mode, which automatically switches after the in-car mode ends. Pet-only mode refers to pet-only mode, which can be activated by the user via voice or manual control. Pet tracking mode refers to pet-tracking mode, which can also be activated by the user via voice or manual control.

[0058] When a person and pet are in the car together, the vehicle's air quality sensor can monitor the air quality in real time, determining if the air is polluted. If it is, the air purification system will automatically activate; if it is not polluted, the air purification system will not activate, and air quality monitoring will continue. The vehicle's camera can also capture images of the pet in real time, identifying the pet's type based on the images and enabling AI interaction with the pet, providing real-time translation and communication. Furthermore, the system can determine the pet's needs based on the collected images and implement solutions to meet those needs.

[0059] When leaving a pet alone in the car, users can activate the "Leave Only" mode via voice or manual operation. Once activated, this mode allows control of window and sunroof locks, as well as gear shift locking, to prevent the pet from leaving the vehicle. It also allows activation of the air conditioning to provide a comfortable environment for the pet, and enables the camera and microphone to send real-time updates on the pet's status. Furthermore, a pet language translation function can be activated to translate pet language in real time.

[0060] When a pet leaves the vehicle alone, a tracking robot installed in the vehicle can track the pet in real time and transmit the pet's status information back to it. Users can speak to their pet through the microphone on the vehicle's infotainment system or a terminal device, and the pet can hear the user's voice through the speaker on the tracking robot.

[0061] S102. Based on the status information of the objects inside the vehicle within a preset time period and the current environmental information inside the vehicle, and based on a preset large model, predict the initial demand information of the objects inside the vehicle, and feed back the initial demand information to the user; wherein, the initial demand information represents the control scheme that the predicted objects inside the vehicle need to be executed by the in-vehicle control equipment.

[0062] For example, a large model is pre-built. This model can be used to predict the needs of objects inside the vehicle. The prediction results of the large model are determined as initial demand information; that is, the large model can output initial demand information. The initial demand information represents the control scheme that the predicted objects inside the vehicle require from the in-vehicle control devices; that is, it predicts how the objects inside the vehicle might want the in-vehicle control devices to work. The in-vehicle control devices are execution modules such as the air conditioner and air purifier in the vehicle. In this embodiment, the model structure of the large model is not specifically limited.

[0063] The system can input the state information of objects inside the vehicle within a preset time period and the current environmental information inside the vehicle into a large model to obtain the initial demand information output by the large model. This initial demand information can then be fed back to the user via the vehicle's infotainment system or a mobile terminal. For example, if the object inside the vehicle is a pet dog, and the state information within the preset time period indicates that the dog is constantly panting with its tongue out, and the current environmental information indicates that the temperature inside the vehicle is 28°C, then the initial demand information for the pet dog can be predicted to be a need for cooling.

[0064] When providing initial requirements to the user, this information can be converted into structured text in a preset format. For example, the structured text could be "The pet is currently showing signs of anxiety" or "The pet is lying down comfortably to rest." This structured text can then be presented to the user, for example, by playing it aloud.

[0065] S103. In response to the user's feedback instruction on the initial demand information, determine the target demand information, and control the corresponding in-vehicle control equipment according to the target demand information.

[0066] For example, after receiving initial request information, the user can determine whether to control the in-vehicle control devices according to the actual situation. For instance, if the initial request information is "lower the interior temperature to 20°C," and the user considers the temperature too low, they can adjust the initial request information and set the adjusted request information as the target request information. That is, the user can provide feedback on the initial request information, achieving timely human-computer interaction. For example, the user can issue feedback commands for the initial request information via voice.

[0067] The user's feedback command can be either to accept or reject the initial request information. Based on the user's feedback command, the target request information can be determined. The target request information is the finally confirmed request of the pet in the vehicle. The in-vehicle control device corresponding to the target request information is identified, and the appropriate plan is executed according to the target request information.

[0068] In-vehicle control devices can include air purifiers, BCM (Body Control Module), AC (Air Conditioner), VCU (Vehicle Control Unit), and robots (tracking robots), etc. Different in-vehicle control devices can be triggered depending on the scenario and needs. Air purifiers may include air quality sensors, air conditioning filters, ion generators, and quantum dot antibacterial material spraying devices, primarily possessing two core functions: air purification and disinfection.

[0069] In scenarios where people and pets coexist, air quality sensors are primarily used to detect the concentration of harmful gases in the vehicle's interior air, determining whether the air is currently polluted. When polluted air is detected, the environmental information on air quality is uploaded to a larger model. For example, if carbon monoxide levels inside the vehicle are detected to be >5 mg / m³... 3 Nitrogen dioxide > 0.085 mg / m³ 3 Ammonia > 15 mg / m³ 3When the system detects that the air inside the vehicle is polluted, a large model can predict initial demand information. If the user agrees to the initial demand information, the air purifier can be controlled based on this information, and the vehicle will actively purify the air. Passengers can also activate the air purifier via voice command; the air purification mode will be activated directly upon receiving the voice command.

[0070] The air conditioning filter removes dust, pet dander, pollen, and other particulate impurities from the air, ensuring clean air. The ion generator decomposes pollutant molecules and neutralizes the charge of suspended particles, causing them to settle naturally and generating a large number of fresh oxygen ions, thus achieving deep purification. The quantum dot antibacterial sprayer eliminates bacteria that may grow in the car due to human-pet interaction, such as E. coli, Salmonella, and Staphylococcus aureus, effectively protecting the hygiene and safety of the in-car environment. The air quality sensor continuously monitors the air quality inside the car and automatically shuts off the air purification mode when the air quality meets standards.

[0071] In scenarios where people and pets are separated, with only the pet remaining in the car, the pet-only mode can be activated manually or via voice commands. When the user activates the pet-only mode, the vehicle's infotainment system sends a signal to the BCM, AC, and VCU, while simultaneously activating the in-vehicle camera and microphone. The data collected by the camera and microphone is then fed back to the T-BOX (TelematicBOX, in-vehicle terminal), allowing the user to monitor the pet's condition in real time.

[0072] When the BCM receives a command to activate the pet-only mode, it issues commands to raise all four windows. After the user leaves the vehicle and locks the doors, it disables the central locking buttons, interior window buttons, and rearview mirror buttons. Simultaneously, it sends a signal indicating successful activation of the pet-only mode to the vehicle's infotainment system, VCU, AC, and T-BOX. Upon receiving this signal, the AC adjusts the air conditioning temperature to a preset level, for example, 23°C. The VCU, upon receiving this signal, prevents the vehicle from depressurizing and disables the gear shift buttons inside the vehicle to prevent accidental gear changes by the pet. When the T-BOX receives this signal, it begins receiving data from the in-vehicle camera and microphone, which is then transmitted to the user. Upon receiving a legitimate unlock request, the BCM sends a signal to exit pet-only mode to the IVI, VCU, AC, and T-BOX, returning to normal operation. Figure 4 This is a diagram illustrating the working process of the in-vehicle control equipment, which enables automatic control of various in-vehicle control devices.

[0073] In this embodiment, the feedback instruction indicates whether the user accepts or does not accept the initial demand information; in response to the user's feedback instruction on the initial demand information, the target demand information is determined, including: if the user's feedback instruction on the initial demand information indicates that the user accepts the initial demand information, then the target demand information is determined to be the initial demand information.

[0074] Specifically, if the user deems the initial requirement information acceptable, they can issue a voice feedback command, "You can begin execution." This signifies the user's acceptance of the initial requirement information, which is then designated as the target requirement. The corresponding in-vehicle control device is then identified, and control is applied according to the initial requirement to execute the plan outlined. For example, if the initial requirement is "lower the temperature to 26°C," the vehicle's air conditioning will be controlled to lower the temperature to 26°C.

[0075] The advantage of this setup is that it eliminates the need for users to observe the state of objects inside the vehicle and predict their needs. Instead, it uses a large model for prediction, automatically controlling the devices within the vehicle and improving vehicle control efficiency. It also prevents user distraction while driving, provides a simple and convenient user interaction method, and enhances the driving experience for both the user and the objects inside the vehicle.

[0076] In this embodiment, determining the target requirement information in response to the user's feedback instruction on the initial requirement information includes: if the user's feedback instruction on the initial requirement information indicates that the user does not accept the initial requirement information, then receiving the requirement adjustment information sent by the user; wherein, the requirement adjustment information indicates the user's adjustment request for the initial requirement information; and determining the target requirement information based on the requirement adjustment information and the initial requirement information.

[0077] Specifically, if a user believes there is a problem with the initial demand information, they can issue a feedback command of "do not execute," in which case the in-vehicle control equipment will not perform any operation, meaning the pet's needs will not be addressed. If the user's feedback command requests an adjustment to the initial demand information, the content of the user's feedback command will be defined as the demand adjustment information. The demand adjustment information represents the user's requirement to adjust the initial demand information; that is, the initial demand information will be adjusted according to the demand adjustment information. The adjusted initial demand information can be defined as the target demand information, and the in-vehicle control equipment will execute the corresponding plan according to the target demand information.

[0078] Users can directly provide specific details of their adjustment requests. For example, if the initial request is "lower the temperature to 20℃," the user could suggest "lowering it to 26℃ would be sufficient." Alternatively, users can interact with the large model to inquire about better solutions. For instance, if the initial request is "open the rear window for ventilation," but the user is on a highway and cannot open the rear window, they could suggest "If it's inconvenient to open the rear window, will adjusting the air conditioning temperature to below 25℃ and turning on the air purifier help?" The large model receives the user's adjustment requests and, based on the adjustment, initial request, current environment, and status information, further considers and provides new request information to the user. This interaction can continue until the user is satisfied with the large model's prediction and action plan, at which point the model's final response is considered the target request. In this embodiment, the semantic understanding and reasoning process during the interaction between the large model and the user is not specifically limited.

[0079] The advantage of this setup is that if the user does not agree with the initial requirements information, they can indicate the parts that need to be adjusted. Through human-computer interaction with the large model, the target requirements information can be obtained, reducing user operations and improving the efficiency of vehicle control.

[0080] In this embodiment, it also includes: associating and storing the current state information of the objects inside the vehicle, the current environmental information inside the vehicle, and the target requirement information.

[0081] Specifically, after determining the target requirement information, the current state information of objects inside the vehicle, the current environment information inside the vehicle, and the target requirement information can be associated and stored. That is, the control scheme preferred by the user in the current situation can be recorded. For example, a preference database can be set up in advance, storing the current state information of objects inside the vehicle, the current environment information inside the vehicle, and the target requirement information in the preference database.

[0082] When predicting demand information using a large model, the state information of objects inside the vehicle within a preset time period, the current environmental information inside the vehicle, and data from a preference database can be input into the large model. That is, the initial demand information can be predicted by combining user preferences. For example, if the preference database contains the state information of objects inside the vehicle at the current moment and the current environmental information inside the vehicle, the demand information corresponding to this state information and current environmental information can be found and used as the predicted initial demand information.

[0083] The benefit of this setup is that it organizes the interaction information between the large model and the user to form a user's own preference database. This preference database provides the large model with user preference data, so that when the same task is encountered again, the model can provide the driver with a better solution based on the driver's preferences, thereby improving the accuracy and efficiency of vehicle control and enhancing the user experience.

[0084] This invention provides a vehicle control method based on a large model. Various information acquisition devices are installed on the vehicle to obtain the state information of objects inside the vehicle and the current environmental information within the vehicle. A large model is pre-constructed. Based on the state information of objects inside the vehicle over a period of time and the current environmental information, the pre-built large model can predict what operations the objects inside the vehicle might expect from the in-vehicle control devices. The predicted results are used as the initial demand information of the objects inside the vehicle and fed back to the user. The user provides feedback on the initial demand information, and the method responds to the user's feedback command to determine the target demand information. Based on the target demand information, the corresponding in-vehicle control devices are controlled to perform the appropriate operations. This method enables timely prediction of the needs of objects inside the vehicle, allowing the in-vehicle control devices to make targeted adjustments, avoiding user distraction while driving, and improving vehicle control accuracy and efficiency. Through human-computer interaction with the user, the method can meet the user's actual needs, improving the driving experience and safety.

[0085] Figure 5 This is a flowchart illustrating a vehicle control method based on a large model, which is an optional embodiment based on the above embodiments.

[0086] In this embodiment, based on the state information of the object inside the vehicle within a preset time period and the current environmental information inside the vehicle, and based on a preset large model, the initial demand information of the object inside the vehicle is predicted. This includes: determining the state change value of the object inside the vehicle based on the state information of the object inside the vehicle within the preset time period; wherein, the state change value characterizes the degree of state change of the object inside the vehicle within the preset time period; if the state change value meets the preset change condition, then based on the state information of the object inside the vehicle at the current moment and the current environmental information inside the vehicle, and based on the preset large model, the initial demand information of the object inside the vehicle is predicted.

[0087] like Figure 5 As shown, the method includes the following steps:

[0088] S501. Using information collection equipment installed on the vehicle, determine the status information of objects inside the vehicle and the current environmental information inside the vehicle within a preset time period; wherein, the status information represents the performance status of objects inside the vehicle.

[0089] For example, this step can refer to step S101 above, and will not be repeated here.

[0090] S502. Based on the state information of the objects inside the vehicle within a preset time period, determine the state change value of the objects inside the vehicle; wherein, the state change value represents the degree of state change of the objects inside the vehicle within the preset time period.

[0091] For example, the state information of an object inside the vehicle is determined within a preset time period, such as determining video data of the object inside the vehicle over 10 minutes. Based on the state information within the preset time period, the state changes of the object inside the vehicle are determined, i.e., the state change values ​​of the object inside the vehicle are obtained. For example, the changes in the object's actions can be determined based on video data, or the changes in the content or volume of the object's speech can be determined based on voice data.

[0092] In this embodiment, the information acquisition device is an image acquisition device, and the status information is image data. Based on the status information of the object inside the vehicle within a preset time period, the status change value of the object inside the vehicle is determined, including: determining a first image and a second image from the image data of the object inside the vehicle within the preset time period; wherein the acquisition time of the first image is earlier than the acquisition time of the second image; and determining the displacement information of the object inside the vehicle based on the first image and the second image, and determining the displacement information as the status change value.

[0093] Specifically, the information acquisition device can be an image acquisition device such as a camera, which can capture images or videos of objects inside the vehicle. That is, the acquired status information can be data in the form of images. The status information within a preset time period includes multiple frames of image data.

[0094] The first image and the second image can be determined from multiple frames of image data collected within a preset time period. The first image is collected earlier than the second image. For example, the first frame of the multi-frame image data can be used as the first image, and the last frame of the multi-frame image data can be used as the second image.

[0095] Image recognition processing is performed on the first and second images respectively to determine the position of the object inside the vehicle in the first image, designated as the first position, and the position of the object inside the vehicle in the second image, designated as the second position. Alternatively, the position of a specific part of the object inside the vehicle in the first image can be identified, designated as the first position, and the position of that specific part of the object inside the vehicle in the second image can be identified, designated as the second position. For example, the position of a pet dog's head can be determined. In this embodiment, the image recognition processing method is not specifically limited. For example, a preset image processing model can be used to determine the position. Based on the first and second positions, the displacement information of the object inside the vehicle can be determined, and the displacement information can characterize the movement of the object inside the vehicle. The displacement information is determined as the state change value of the object inside the vehicle.

[0096] The advantage of this setup is that by collecting image data at different times, the range of motion of objects inside the vehicle can be determined, thereby obtaining the state change values ​​of objects inside the vehicle, achieving precise monitoring of objects inside the vehicle, and improving the accuracy of subsequent vehicle control.

[0097] In this embodiment, the method further includes: performing image recognition processing on the first image and the second image respectively to obtain the outline information of the object inside the vehicle in the first image and the outline information of the object inside the vehicle in the second image; and determining the outline information of the object inside the vehicle in the first image and the outline information of the object inside the vehicle in the second image as the state change value of the object inside the vehicle.

[0098] Specifically, after determining the first image and the second image, image recognition processing can be performed on the contours of the objects inside the vehicle in the first image and the second image respectively to obtain the contour information of the objects inside the vehicle in the first image and the second image. In this embodiment, the method of contour recognition is not specifically limited. The contour information of the objects inside the vehicle in the first image and the second image are determined as the state change values ​​of the objects inside the vehicle; that is, the state change values ​​may include the contour information of the objects inside the vehicle in the first image and the second image.

[0099] The advantage of this setup is that by determining the outline information of objects inside the vehicle, we can obtain the state of those objects. For example, we can determine whether an object inside the vehicle is always in a curled-up state, which makes it easier to accurately judge the situation of the object inside the vehicle and improve the accuracy of subsequent vehicle control.

[0100] In this embodiment, the information acquisition device is a sound acquisition device, and the status information is audio data. Based on the status information of the object inside the vehicle within a preset time period, the status change value of the object inside the vehicle is determined, including: determining the decibel value corresponding to the preset time period based on the audio data of the object inside the vehicle within the preset time period; determining the decibel range of the decibel value; and determining the sound state corresponding to the decibel range based on a preset correlation relationship. The preset correlation relationship represents the correlation between the decibel range and the sound state. The sound state corresponding to the decibel range is then determined as the status change value of the object inside the vehicle.

[0101] Specifically, the information collection device can be a microphone or other sound collection device that can collect audio data of objects inside the vehicle. The audio data of objects inside the vehicle within a preset time period is acquired, and based on this audio data, the decibel value of the sound emitted by the objects inside the vehicle within that preset time period is determined.

[0102] Multiple decibel ranges are pre-defined to determine which decibel range an object inside the vehicle falls within, i.e., to identify the decibel range in which the decibel value falls. Various pre-defined correlations exist between decibel ranges and sound states. Sound states characterize the state of the sound emitted by an object inside the vehicle; for example, a short bark, a continuous whimper, or a high-decibel bark. Based on the pre-defined correlations, the system finds the sound state corresponding to the decibel range in which the decibel value falls and identifies that sound state as the state change value of the object inside the vehicle.

[0103] The advantage of this setup is that it allows for the monitoring of sounds from objects inside the vehicle, enabling comprehensive monitoring of status information, improving the accuracy of demand information prediction, and thus enhancing vehicle control precision.

[0104] S503. If the state change value meets the preset change conditions, then based on the state information of the object inside the vehicle at the current moment and the current environmental information inside the vehicle, the initial demand information of the object inside the vehicle is predicted based on the preset large model.

[0105] For example, pre-set change conditions can be used to determine whether the state of objects inside the vehicle is abnormal. For instance, preset change conditions might include a continuous displacement of the object inside the vehicle greater than 500mm, physical abnormalities such as curling up or trembling, or continuous high-decibel barking. The system determines whether the state change information meets the preset change conditions. If not, the object inside the vehicle is considered to be in good condition, and there is no need to predict its needs; the vehicle's control equipment is operating normally. If the conditions are met, the object inside the vehicle is considered to be abnormal, and the current state information of the object inside the vehicle and the current environmental information inside the vehicle need to be input into the large model to predict the object's initial needs.

[0106] For example, a state change value is the displacement information of an object inside the vehicle. If the displacement of the object exceeds 500mm, the object is considered abnormal. State change values ​​can include various information such as displacement, contour, and sound status. Each type of information can have its own change conditions. An object can be considered abnormal only when all state change values ​​meet their corresponding conditions, or when one or more state change values ​​meet their corresponding conditions.

[0107] In this embodiment, if the state change value meets the preset change conditions, then based on the state information of the object inside the vehicle at the current moment and the current environmental information inside the vehicle, and based on the preset large model, the initial demand information of the object inside the vehicle is predicted. This includes: if the state change value meets the preset change conditions, then the state change value is fed back to the user; wherein, the state change value is used to prompt the user to confirm whether the demand of the object inside the vehicle needs to be predicted; in response to the user's confirmation prediction instruction, based on the state information of the object inside the vehicle at the current moment and the current environmental information inside the vehicle, and based on the preset large model, the initial demand information of the object inside the vehicle is predicted.

[0108] Specifically, if the state change value meets the preset change conditions, the initial demand information of the objects inside the vehicle can be predicted directly based on the current state information of the objects inside the vehicle and the current environmental information inside the vehicle. This initial demand information is then fed back to the user, who can then confirm whether they accept it.

[0109] Alternatively, after confirming that the state change value meets preset change conditions, the state change value can be fed back to the user. Upon receiving the state change value, the user can confirm whether it's necessary to predict the needs of objects inside the vehicle. For example, if the state change value indicates that an object inside the vehicle is rapidly wagging its tail, the user might consider this a normal state for the object and not an abnormal one; therefore, predicting the object's needs is unnecessary. If the user believes that predicting the object's needs is unnecessary, they can issue a command to not perform a prediction via voice or other means. Upon receiving the command, the vehicle's infotainment system will not perform a prediction and will continue monitoring the state information of objects inside the vehicle.

[0110] If the user confirms an anomaly in an object inside the vehicle based on the state change value, they can issue a confirmation command via voice or other means. Upon receiving the confirmation command, the vehicle's infotainment system inputs the current state information of the object inside the vehicle and the current environmental information into the large model to obtain the initial requirement information for the object.

[0111] In other words, the state change value can serve as an early warning of abnormalities in objects inside the vehicle. After receiving the warning, the user can pull over and check the pet's real-time status, and then handle the situation promptly based on their own experience. Alternatively, the user can choose not to use the large model for demand prediction, or they can initiate a deep consultation with the large model to request it to predict the needs of objects inside the vehicle, and then handle the situation based on the prediction results.

[0112] The advantage of this setup is that when a user receives an abnormal warning about an object inside the vehicle, they can determine whether to initiate a deeper consultation with the large model based on the actual situation, thereby improving the flexibility of demand forecasting and enhancing the human-computer interaction experience.

[0113] In this embodiment, based on the current state information of the object inside the vehicle and the current environment information inside the vehicle, the initial demand information of the object inside the vehicle is predicted based on a preset large model. This includes: retrieving pre-stored consultation information from a preset cloud platform information database; inputting the consultation information, the current state information of the object inside the vehicle, and the current environment information inside the vehicle into the preset large model to obtain the output initial demand information of the object inside the vehicle.

[0114] Specifically, the large-scale model can predict demand based on two sets of data: the current state of objects inside the vehicle and the current environmental information within the vehicle. Alternatively, a pre-built cloud platform can be used, allowing the large-scale model to access data from it. This cloud platform contains a pre-built database storing a large amount of information, such as veterinary medical literature, pet case records, and pet behavioral characteristics. When applying the large-scale model for prediction, the input data can be three sets of data: the information from the cloud platform, the current state of objects inside the vehicle, and the current environmental information within the vehicle. Based on this three sets of data, the large-scale model derives initial demand information.

[0115] The advantage of this setup is that by combining data from the cloud platform, the inference accuracy of large models can be improved, that is, the prediction accuracy of initial demand information can be improved.

[0116] Figure 6 This is a flowchart of the workflow for a large model. Figure 6 In this context, information acquisition sensors refer to sensors used to collect information about a pet's language, posture, and movements; for example, cameras and microphones. In-vehicle cameras primarily monitor the status of occupants and pets inside the vehicle, capturing and recording multi-dimensional information such as the pet's location, sound signals, body posture, and facial expressions in real time. When the in-vehicle camera detects a pet entering the vehicle, it begins scanning the pet's features to obtain status information such as the pet's voice, posture, and movements.

[0117] Other sensors inside the vehicle may include sound sensors, temperature sensors, air sensors, seat pressure sensors, etc., for comprehensively collecting and analyzing the in-vehicle environment.

[0118] Model A refers to a model that identifies and processes the pet's state information. This could be an image recognition model or a sound recognition model, used to determine the state information. Specifically, when the pet is in the car, the camera can capture images of the pet at certain intervals and frequencies. An algorithm that recognizes continuous pet movements compares two consecutive images. If the state change value reaches a threshold, the state information, current environmental information, and information from the cloud platform are transmitted to Model B. Model B is a pre-set, large-scale model used to predict initial demand information.

[0119] Once the state change value reaches a critical threshold, an anomaly warning can be pushed to the car owner, i.e., the pet owner. Upon receiving the warning from Model A, the pet owner can immediately pull over and check the pet's real-time status, taking appropriate action based on their experience. Alternatively, they can consult with Model B for further action.

[0120] Model B uses triple data fusion for inference and prediction, and provides corresponding suggested solutions to pet owners, i.e., outputting initial demand information. Pet owners can determine whether they accept the initial demand information predicted by Model B. If the pet owner is satisfied with the initial demand information and accepts the suggestions, the corresponding instructions are automatically generated and executed through in-vehicle execution modules, such as the air conditioning, robots, and vehicle body controllers. For example, if the model detects the pet's restlessness and, after diagnosis, determines that the pet may be experiencing motion sickness, it suggests lowering the rear window halfway for ventilation. If the pet owner agrees, the vehicle body sensors are instructed to open the window. If the pet owner is not satisfied with the initial demand information or wishes to adjust it, the feedback can be re-input into Model B. Model B, based on the new input information, re-analyzes and outputs an updated next action plan, which is then fed back to the pet owner. This process continues until the pet owner is satisfied with the final result, which is the target demand information. For example, if a pet is detected as restless, Model B might diagnose that the pet is experiencing motion sickness and suggests lowering the rear window halfway for ventilation. The pet owner might respond, "We're currently on the highway, so it's inconvenient to open the rear window. Would adjusting the air conditioning to below 25°C and turning on the air purifier help?" If Model B believes the pet is exhibiting more serious symptoms, it might also recommend that the pet owner take the pet to a veterinary hospital immediately.

[0121] Model B possesses a feedback and adaptive optimization mechanism. After each round of questioning, it can organize environmental information collected by in-vehicle sensors, the execution status of in-vehicle execution modules, pet status information, and interaction information between Model B and the car owner to form its own preference database. This preference database can provide Model B with user preference data, allowing it to offer a better solution to the car owner based on those preferences when encountering the same task again.

[0122] S504. In response to the user's feedback instruction on the initial demand information, determine the target demand information, and control the corresponding in-vehicle control equipment according to the target demand information.

[0123] For example, this step can refer to step S103 above, and will not be repeated here.

[0124] This invention provides a vehicle control method based on a large model. Various information acquisition devices are installed on the vehicle to obtain the state information of objects inside the vehicle and the current environmental information within the vehicle. A large model is pre-constructed. Based on the state information of objects inside the vehicle over a period of time and the current environmental information, the pre-built large model can predict what operations the objects inside the vehicle might expect from the in-vehicle control devices. The predicted results are used as the initial demand information of the objects inside the vehicle and fed back to the user. The user provides feedback on the initial demand information, and the method responds to the user's feedback command to determine the target demand information. Based on the target demand information, the corresponding in-vehicle control devices are controlled to perform the appropriate operations. This method enables timely prediction of the needs of objects inside the vehicle, allowing the in-vehicle control devices to make targeted adjustments, improving vehicle control accuracy and efficiency. Through human-computer interaction with the user, the method can meet the user's actual needs, further improving vehicle control accuracy and enhancing the driving experience.

[0125] Figure 7 This is a structural block diagram of a vehicle control device based on a large model, provided as an embodiment of the present invention. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. (Refer to...) Figure 7 The vehicle control device 700 based on a large model includes: an information determination module 701, a demand prediction module 702, and a vehicle control module 703.

[0126] The information determination module 701 is used to determine the status information of objects inside the vehicle and the current environmental information inside the vehicle within a preset time period through information collection equipment installed on the vehicle; wherein, the status information represents the performance status of the objects inside the vehicle.

[0127] The demand prediction module 702 is used to predict the initial demand information of the in-vehicle object based on the state information of the in-vehicle object within a preset time period and the current environmental information inside the vehicle, and to feed back the initial demand information to the user; wherein, the initial demand information represents the control scheme that the predicted in-vehicle object needs to be executed by the in-vehicle control device.

[0128] The vehicle control module 703 is used to respond to the user's feedback command on the initial demand information, determine the target demand information, and control the corresponding in-vehicle control equipment according to the target demand information.

[0129] Figure 8 A structural block diagram of a vehicle control device based on a large model provided in an embodiment of the present invention is shown below. Figure 8As shown, the vehicle control device 800 based on a large model includes an information determination module 801, a demand prediction module 802, and a vehicle control module 803. The demand prediction module 802 includes a state change unit 8021 and an initial determination unit 8022.

[0130] The state change unit 8021 is used to determine the state change value of the object inside the vehicle based on the state information of the object inside the vehicle within a preset time period; wherein the state change value characterizes the degree of state change of the object inside the vehicle within the preset time period.

[0131] The initial determination unit 8022 is used to predict the initial demand information of the in-vehicle object based on the current state information of the in-vehicle object and the current environmental information inside the vehicle, according to a preset large model, if the state change value meets the preset change conditions.

[0132] In one example, the information acquisition device is an image acquisition device, and the status information is image data; the status change unit 8021 is specifically used for:

[0133] From the image data of the object inside the vehicle within a preset time period, a first image and a second image are determined; wherein the acquisition time of the first image is earlier than the acquisition time of the second image;

[0134] Based on the first image and the second image, the displacement information of the object inside the vehicle is determined, and the displacement information is determined as the state change value.

[0135] In one example, the state transition unit 8021 is also specifically used for:

[0136] Image recognition processing is performed on the first image and the second image respectively to obtain the outline information of the objects inside the vehicle in the first image and the outline information of the objects inside the vehicle in the second image;

[0137] The outline information of the objects inside the vehicle in the first image and the outline information of the objects inside the vehicle in the second image are determined as the state change values ​​of the objects inside the vehicle.

[0138] In one example, the information acquisition device is a sound acquisition device, and the status information is audio data; the status change unit 8021 is specifically used for:

[0139] Based on the audio data of the object inside the vehicle within a preset time period, determine the decibel value corresponding to the preset time period;

[0140] The decibel range of the decibel value is determined, and the sound state corresponding to the decibel range is determined according to a preset correlation relationship; wherein, the preset correlation relationship represents the correlation between the decibel range and the sound state;

[0141] The sound state corresponding to the decibel range is determined as the state change value of the object inside the vehicle.

[0142] In one example, the initial determination unit 8022 is specifically used for:

[0143] If the state change value meets the preset change conditions, the state change value is fed back to the user; wherein, the state change value is used to prompt the user to confirm whether it is necessary to predict the needs of objects inside the vehicle.

[0144] In response to the user's confirmation prediction command, based on the current state information of the objects inside the vehicle and the current environmental information inside the vehicle, the system predicts the initial demand information of the objects inside the vehicle using a preset large model.

[0145] In one example, the initial determination unit 8022 is specifically used for:

[0146] Retrieve pre-stored consultation information from the preset cloud platform database;

[0147] The consultation information, the current status information of the objects inside the vehicle, and the current environmental information inside the vehicle are input into a preset large model to obtain the initial demand information of the objects inside the vehicle.

[0148] In one example, the feedback instruction indicates whether the user accepts or does not accept the initial request information; the vehicle control module 803 includes:

[0149] The first feedback unit is configured to determine the target demand information as the initial demand information if the user's feedback instruction on the initial demand information indicates that the user accepts the initial demand information.

[0150] In one example, the vehicle control module 803 includes:

[0151] The second feedback unit is configured to receive demand adjustment information sent by the user if the user's feedback instruction on the initial demand information indicates that the user does not accept the initial demand information; wherein the demand adjustment information indicates the user's adjustment request for the initial demand information.

[0152] The demand adjustment unit is used to determine the target demand information based on the demand adjustment information and the initial demand information.

[0153] One example also includes:

[0154] The storage module is used to associate and store the current state information of the objects inside the vehicle, the current environmental information inside the vehicle, and the target requirement information.

[0155] One example also includes:

[0156] A category determination module is used to determine the object category of the objects inside the vehicle.

[0157] A button blocking module is used to adjust the preset buttons in the vehicle to a blocking mode when the object category of the in-vehicle object is a preset category and only the in-vehicle object exists in the vehicle; wherein, the blocking mode indicates that the button will not be triggered by pressing operation.

[0158] In one example, the vehicle is equipped with a tracking robot, which is also equipped with information collection devices; the device further includes:

[0159] The tracking request module is used to send a tracking request to the user when the object inside the vehicle leaves the vehicle if the object category is a preset category; wherein, the tracking request is used to prompt the user to confirm whether to track the object inside the vehicle through the tracking robot;

[0160] The robot tracking module is used to control the tracking robot to follow the object inside the vehicle and acquire the status information of the object inside the vehicle collected by the tracking robot if it responds to the user's confirmation command for the tracking request.

[0161] The information feedback module is used to provide the status information to the user.

[0162] Figure 9 A structural block diagram of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, the electronic device includes: a memory 91 and a processor 92; the memory 91 is a memory used to store instructions executable by the processor 92.

[0163] The processor 92 is configured to perform the methods provided in the embodiments described above.

[0164] The electronic device also includes a receiver 93 and a transmitter 94. The receiver 93 is used to receive instructions and data sent by other devices, and the transmitter 94 is used to send instructions and data to external devices.

[0165] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment. The device may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, personal digital assistant, etc.

[0166] The device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0167] Processing component 1002 typically controls the overall operation of device 1000, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1002 may include one or more processors 1020 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1002 may include one or more modules to facilitate interaction between processing component 1002 and other components. For example, processing component 1002 may include a multimedia module to facilitate interaction between multimedia component 1008 and processing component 1002.

[0168] The device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0169] Memory 1004 is configured to store various types of data to support the operation of device 1000. Examples of such data include instructions for any application or method operating on device 1000, contact data, phonebook data, messages, pictures, videos, etc. Memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0170] Power supply component 1006 provides power to various components of device 1000. Power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 1000.

[0171] Multimedia component 1008 includes a screen that provides an output interface between the device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 908 includes a front-facing camera and / or a rear-facing camera. When the device 1000 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0172] Audio component 1010 is configured to output and / or input audio signals. For example, audio component 1010 includes a microphone (MIC) configured to receive external audio signals when device 1000 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1004 or transmitted via communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio signals.

[0173] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0174] Sensor assembly 1014 includes one or more sensors for providing state assessments of various aspects of device 1000. For example, sensor assembly 1014 may detect the on / off state of device 1000, the relative positioning of components such as the display and keypad of device 1000, changes in the position of device 1000 or a component of device 1000, the presence or absence of user contact with device 1000, the orientation or acceleration / deceleration of device 1000, and temperature changes of device 1000. Sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1014 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0175] Communication component 1016 is configured to facilitate wired or wireless communication between device 1000 and other devices. Device 1000 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 1016 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1016 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0176] In an exemplary embodiment, the apparatus 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0177] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, which can be executed by a processor 1020 of the device 1000 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0178] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the aforementioned vehicle control method based on a large model.

[0179] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0180] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A vehicle control method based on a large model, characterized in that, include: By using information collection devices installed on the vehicle, the status information of objects inside the vehicle and the current environmental information inside the vehicle are determined within a preset time period; wherein, the status information represents the performance status of the objects inside the vehicle. Based on the state information of the in-vehicle object within a preset time period and the current environmental information inside the vehicle, the initial demand information of the in-vehicle object is predicted based on a preset large model, and the initial demand information is fed back to the user; wherein, the initial demand information represents the control scheme that the predicted in-vehicle object needs to be executed by the in-vehicle control device. In response to the user's feedback instruction on the initial demand information, the target demand information is determined, and the corresponding in-vehicle control equipment is controlled according to the target demand information.

2. The method according to claim 1, characterized in that, Based on the state information of the objects inside the vehicle within a preset time period and the current environmental information inside the vehicle, and based on a preset large model, the initial demand information of the objects inside the vehicle is predicted, including: Based on the state information of the object inside the vehicle within a preset time period, the state change value of the object inside the vehicle is determined; wherein, the state change value characterizes the degree of state change of the object inside the vehicle within the preset time period; If the state change value meets the preset change conditions, then based on the state information of the object inside the vehicle at the current moment and the current environmental information inside the vehicle, the initial demand information of the object inside the vehicle is predicted according to the preset large model.

3. The method according to claim 2, characterized in that, The information acquisition device is an image acquisition device, and the status information is image data; Based on the state information of the objects inside the vehicle within a preset time period, determine the state change value of the objects inside the vehicle, including: From the image data of the object inside the vehicle within a preset time period, a first image and a second image are determined; wherein the acquisition time of the first image is earlier than the acquisition time of the second image; Based on the first image and the second image, the displacement information of the object inside the vehicle is determined, and the displacement information is determined as the state change value.

4. The method according to claim 3, characterized in that, Also includes: Image recognition processing is performed on the first image and the second image respectively to obtain the outline information of the objects inside the vehicle in the first image and the outline information of the objects inside the vehicle in the second image; The outline information of the objects inside the vehicle in the first image and the outline information of the objects inside the vehicle in the second image are determined as the state change values ​​of the objects inside the vehicle.

5. The method according to claim 2, characterized in that, The information acquisition device is a sound acquisition device, and the status information is audio data; Based on the state information of the objects inside the vehicle within a preset time period, determine the state change value of the objects inside the vehicle, including: Based on the audio data of the object inside the vehicle within a preset time period, determine the decibel value corresponding to the preset time period; The decibel range of the decibel value is determined, and the sound state corresponding to the decibel range is determined according to a preset correlation relationship; wherein, the preset correlation relationship represents the correlation between the decibel range and the sound state; The sound state corresponding to the decibel range is determined as the state change value of the object inside the vehicle.

6. The method according to claim 2, characterized in that, If the state change value meets the preset change conditions, then based on the current state information of the object inside the vehicle and the current environmental information inside the vehicle, and based on a preset large model, the initial demand information of the object inside the vehicle is predicted, including: If the state change value meets the preset change conditions, the state change value is fed back to the user; wherein, the state change value is used to prompt the user to confirm whether it is necessary to predict the needs of objects inside the vehicle. In response to the user's confirmation prediction command, based on the current state information of the objects inside the vehicle and the current environmental information inside the vehicle, the system predicts the initial demand information of the objects inside the vehicle using a preset large model.

7. The method according to claim 6, characterized in that, Based on the current state information of the objects inside the vehicle and the current environmental information inside the vehicle, and based on a preset large model, the initial demand information of the objects inside the vehicle is predicted, including: Retrieve pre-stored consultation information from the preset cloud platform database; The consultation information, the current status information of the objects inside the vehicle, and the current environmental information inside the vehicle are input into a preset large model to obtain the initial demand information of the objects inside the vehicle.

8. The method according to claim 1, characterized in that, The feedback instruction indicates whether the user accepts or does not accept the initial request information; In response to the user's feedback instruction on the initial requirement information, the target requirement information is determined, including: If the user's feedback instruction on the initial requirement information indicates that the user accepts the initial requirement information, then the target requirement information is determined to be the initial requirement information.

9. The method according to claim 8, characterized in that, In response to the user's feedback instruction on the initial requirement information, the target requirement information is determined, including: If the user's feedback instruction on the initial requirement information indicates that the user does not accept the initial requirement information, then the user's requirement adjustment information is received; wherein, the requirement adjustment information indicates the user's request to adjust the initial requirement information; Based on the demand adjustment information and the initial demand information, the target demand information is determined.

10. The method according to claim 1, characterized in that, Also includes: The current state information of the objects inside the vehicle, the current environmental information inside the vehicle, and the target requirement information are associated and stored.

11. The method according to any one of claims 1-10, characterized in that, Also includes: Determine the object category of the objects inside the vehicle; If the object category of the in-vehicle object is a preset category, then when only the in-vehicle object exists in the vehicle, the preset button in the vehicle will be adjusted to a shielded mode; wherein, the shielded mode indicates that the button will not be triggered by pressing operation.

12. The method according to claim 11, characterized in that, The vehicle is equipped with a tracking robot, which is equipped with information collection equipment; the method further includes: If the object category of the object inside the vehicle is a preset category, a tracking request is sent to the user when the object leaves the vehicle; wherein, the tracking request is used to prompt the user to confirm whether to track the object inside the vehicle through a tracking robot; If a confirmation command is received from the user regarding the tracking request, the tracking robot is controlled to follow the object inside the vehicle and the status information of the object inside the vehicle collected by the tracking robot is obtained. The status information is then fed back to the user.

13. A vehicle control device based on a large model, characterized in that, include: The information determination module is used to determine the status information of objects inside the vehicle and the current environmental information inside the vehicle within a preset time period through information collection devices installed on the vehicle; wherein, the status information represents the performance status of the objects inside the vehicle. The demand prediction module is used to predict the initial demand information of the in-vehicle objects based on the state information of the objects in the vehicle within a preset time period and the current environmental information inside the vehicle, and to feed back the initial demand information to the user; wherein, the initial demand information represents the control scheme that the predicted in-vehicle objects need to be executed by the in-vehicle control equipment. The vehicle control module is used to respond to the user's feedback command on the initial demand information, determine the target demand information, and control the corresponding in-vehicle control equipment according to the target demand information.

14. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-12.

16. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-12.

Images