Vehicle-mounted scene-oriented mobile image equipment control method and equipment and storage medium
By analyzing the perspective of the gimbal of the on-board mobile image device, the problem of complex and error of manual control in on-board scenes is solved, and accurate and safe viewing angle control and consistent shooting are achieved.
Patent Information
- Application Number
- CN202510506519.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-22
- Publication Date
- 2025-07-18
AI Technical Summary
In car-mounted scenarios, the manual control of mobile imaging devices is complex and has errors, making it difficult to accurately reproduce a fixed viewing angle, affecting driving safety and shooting consistency.
By obtaining the viewing angle control request, analyzing the target gimbal viewing angle mark, determining the gimbal viewing angle information, and automatically adjusting the gimbal viewing angle of the on-board mobile image device, supporting a variety of user input methods and emotional state detection, and automatically adjusting the gimbal viewing angle based on the on-board map information and camera environment information.
It realizes accurate adjustment of the gimbal angle of the on-board mobile imaging equipment, ensures the consistency of the viewing angle for each shooting, reduces driving risks and operation complexity, and improves the consistency and safety of the shooting effect.
Smart Images
Figure CN120343404A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of in-vehicle mobile imaging, and particularly to a control method, device, and storage medium for mobile imaging devices in in-vehicle scenarios. Background Art
[0002] In recent years, the application of mobile imaging devices such as action cameras, handheld gimbal cameras, and portable panoramic cameras in in-vehicle scenarios has become increasingly popular. During driving, users often expect to not only record the driving process through the shooting device but also capture occasional scenery and road condition details.
[0003] When using a mobile imaging device in a vehicle, users need to use a corresponding vehicle-mounted bracket, and the placement location is highly uncertain. It may be inside the vehicle, outside the vehicle, in the front row, or in the back row. Facing such a highly uncertain scenario, it is difficult for users to adjust the gimbal angle by operating the joystick, which may affect driving safety. In addition, users cannot ensure that the shooting angle is the same each time, and it is difficult to accurately manually adjust the joystick to adjust the gimbal angle to achieve the same effect each time.
[0004] In response to the above problems, the industry has not yet proposed a better solution. Summary of the Invention
[0005] Embodiments of this application provide a control method, device, and storage medium for mobile imaging devices in in-vehicle scenarios, which are used to at least solve the problems in traditional in-vehicle mobile imaging recording that manual operation is complex and there are manual operation errors, making it difficult to accurately reproduce a fixed perspective.
[0006] In a first aspect, embodiments of this application provide a control method for a mobile imaging device in an in-vehicle scenario, including: obtaining a perspective control request and parsing a target gimbal perspective identifier corresponding to the perspective control request; determining target gimbal perspective azimuth information that matches the target gimbal perspective identifier; the parameter type of the gimbal perspective azimuth parameter includes at least one of the following: gimbal attitude information, gimbal camera shooting parameters, or gimbal position information; controlling and adjusting the gimbal perspective of the in-vehicle mobile imaging device according to the target gimbal perspective azimuth parameter; the in-vehicle mobile imaging device is a mobile imaging device in an in-vehicle scenario.
[0007] Optionally, the obtaining of the perspective control request includes: obtaining user input information and detecting whether the user input information includes a gimbal perspective identifier; the user input information includes any one of the following: graphical interface input information, voice input information, gesture input information, button input information, or joystick input information; when it is detected that the user input information includes a gimbal perspective identifier, a perspective control request is generated according to the gimbal perspective identifier included in the user input information.
[0008] Optionally, obtaining the user input information and detecting whether the user input information includes a pan-tilt perspective identifier includes: detecting a user trigger operation based on at least one perspective option in a graphical user interface; each of the perspective options is respectively configured to have a corresponding pan-tilt perspective identifier; when it is detected that the user input information includes a pan-tilt perspective identifier, generating a perspective control request according to the pan-tilt perspective identifier included in the user input information, including: when it is detected that there is a user trigger operation for a target perspective option, generating a perspective control request according to the pan-tilt perspective identifier corresponding to the target perspective option.
[0009] Optionally, obtaining the perspective control request includes: obtaining the emotion state detection results of at least one vehicle occupant; when the emotion state detection result of a first vehicle occupant meets a preset emotion condition, determining a first pan-tilt perspective identifier that matches the seating position of the first vehicle occupant; generating a perspective control request based on the first pan-tilt perspective identifier.
[0010] Optionally, obtaining the perspective control request includes: obtaining vehicle-mounted map information and detecting whether there is a user interest point near the vehicle according to the vehicle-mounted map information, or obtaining the vehicle environment information captured by a vehicle-mounted camera and detecting whether there is a user interest point near the vehicle according to the vehicle environment information; when there is the user interest point, determining a matching pan-tilt perspective identifier according to the relative orientation between the vehicle and the user interest point; generating a perspective control request based on the second pan-tilt perspective identifier.
[0011] Optionally, determining the target pan-tilt perspective orientation information that matches the target pan-tilt perspective identifier includes: determining the target pan-tilt perspective orientation information that matches the target pan-tilt perspective identifier according to a pan-tilt perspective correspondence; the pan-tilt perspective correspondence defines the association relationship between the pan-tilt perspective identifier and the pan-tilt perspective orientation parameter; for the setting of the pan-tilt perspective correspondence, it includes: obtaining a pan-tilt perspective entry request and parsing the pan-tilt perspective entry identifier and the pan-tilt perspective orientation entry parameter of the vehicle-mounted mobile imaging device corresponding to the pan-tilt perspective entry request; setting the pan-tilt perspective correspondence according to the pan-tilt perspective entry identifier and the pan-tilt perspective orientation entry parameter.
[0012] Optionally, obtaining the pan-tilt perspective entry request includes: when it is detected that there is user manual adjustment information for the pan-tilt perspective orientation parameter of the vehicle-mounted mobile imaging device, generating a pan-tilt perspective save notification; in the case of detecting a confirmation feedback from the user for the pan-tilt perspective save notification, generating a new pan-tilt perspective identifier, and generating the pan-tilt perspective entry request in combination with the pan-tilt perspective orientation parameter that is adjusted and updated based on the user manual adjustment information.
[0013] In a second aspect, an embodiment of the present application provides a storage medium storing one or more programs including execution instructions that can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) for executing any one of the above-mentioned mobile imaging device control methods for vehicle-mounted scenarios of the present application.
[0014] In a third aspect, an electronic device is provided, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of the above-mentioned mobile imaging device control methods for vehicle-mounted scenarios of the present application.
[0015] In a fourth aspect, an embodiment of the present application further provides a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer is enabled to execute any one of the above-mentioned mobile imaging device control methods for vehicle-mounted scenarios.
[0016] The beneficial effects of the embodiments of the present application are as follows: By obtaining a perspective control request and parsing the target pan-tilt perspective identifier, and determining the pan-tilt perspective azimuth information in combination with the target pan-tilt perspective identifier, the pan-tilt perspective of the mobile imaging device in the vehicle-mounted scenario is accurately adjusted. By replacing manual operation with intelligent control, the errors and instability caused by traditional manual adjustment are avoided. In addition, by calling the associated pan-tilt perspective azimuth information using the pan-tilt perspective identifier, the consistency of the shooting perspective each time is ensured, and the shooting angle deviation caused by different device positions in the traditional method is avoided. In addition, during driving, the user does not need to manually adjust the pan-tilt, enabling the driver to focus on driving safety and reducing the safety hazards caused by operation errors or distractions. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0018] Figure 1 Shows a flowchart of an example of a mobile imaging device control method for vehicle-mounted scenarios according to an embodiment of the present application; Figure 2 Shows an operation flowchart of an example of obtaining a perspective control request according to an embodiment of the present application; Figure 3 The figure shows a schematic diagram of the interface effect of an example of a graphical interaction interface corresponding to the input information of the graphical interface; Figure 4 The figure shows a flowchart of an example of obtaining a perspective control request according to an embodiment of the present application; Figure 5 The figure shows a flowchart of an example of obtaining a perspective control request according to an embodiment of the present application; Figure 6 The figure shows a schematic diagram of the shooting screen effect of an example of a user's point of interest according to an embodiment of the present application; Figure 7 The figure shows a flowchart of an example of setting the corresponding relationship of the pan-tilt perspective according to an embodiment of the present application; Figure 8 The figure shows a diagram of the interface interaction change of an example of performing a perspective operation on a graphical interface according to an embodiment of the present application; Figure 9 The figure is a schematic structural diagram of an embodiment of an electronic device of the present application. Detailed implementation manners
[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.
[0020] It should also be noted that in this text, the terms "including" and "comprising" not only include those elements, but also include other elements not explicitly listed, or elements inherent to such a process, method, article, or device. Without further limitation, the elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article, or device including the said elements.
[0021] In the technical solution of the present application, for the processing of collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved, etc., it complies with the provisions of relevant laws and regulations and does not violate public order and good customs.
[0022] Figure 1 The figure shows a flowchart of an example of a method for controlling a mobile imaging device for an in-vehicle scenario according to an embodiment of the present application.
[0023] Regarding the execution subject of the method in the embodiments of the present application, it can be any controller or processor with computing or processing capabilities, equipped with a mobile imaging device control system for vehicle-mounted scenarios. By combining the perspective control request, it intelligently performs pan-tilt control and precise perspective adjustment on the vehicle-mounted mobile imaging device, not only improving the operation convenience of the vehicle-mounted mobile imaging device, but also effectively enhancing driving safety and the consistency of the shooting perspective.
[0024] In some examples, the method in the embodiments of the present application can be integrated and configured in an electronic device or terminal in a software, hardware, or software-hardware combination manner, and the types of the terminal or electronic device can be diverse, such as mobile phones, tablet computers, desktop computers, or in-vehicle terminal devices, etc.
[0025] As Figure 1 shown, in step S110, a perspective control request is obtained, and the target pan-tilt perspective identifier corresponding to the perspective control request is parsed.
[0026] Here, the system can receive a perspective control request for the vehicle-mounted mobile imaging device sent by a user or other system modules, and can be transmitted in various ways. For example, the user inputs an instruction through an interface, or the system obtains it through automated task scheduling. Furthermore, the target pan-tilt perspective identifier carried in the request is parsed. The information type of the pan-tilt perspective identifier can be diverse. For example, it can be the encoding information (such as a binary or hexadecimal encoded string generated based on the pan-tilt perspective azimuth information) referring to the pan-tilt perspective azimuth information (such as the pitch angle and yaw angle of the pan-tilt), and it can also be some preset perspective names (such as "front perspective", "left front side perspective", "right front side perspective", etc.), and it can also be various perspective names defined by the user (such as "perspective A", "perspective B", etc.).
[0027] It should be understood that the triggering devices of the perspective control request can be diverse. For example, it can be initiated through a vehicle-mounted voice assistant, a mobile application APP, a vehicle-mounted display screen, a joystick, or a physical button, etc., and no restrictions should be made here for the time being. In addition, the vehicle-mounted mobile imaging device is a mobile imaging device in a vehicle-mounted scenario, and can be various consumer-grade handheld imaging devices in a vehicle-mounted usage scenario. The device types of the vehicle-mounted mobile imaging device can also be diverse, such as action cameras, handheld gimbal cameras, or portable panoramic cameras, etc. These devices can be fixed inside or outside the vehicle through a vehicle-mounted bracket, and in the current related technologies, the perspective control of the vehicle-mounted mobile imaging device is generally realized based on a manual operation method.
[0028] In step S120, the target pan-tilt perspective azimuth information matching the target pan-tilt perspective identifier is determined.
[0029] Here, the parameter type of the pan-tilt perspective orientation parameter includes at least one of the following: pan-tilt attitude information, pan-tilt camera shooting parameters, or pan-tilt position information. Exemplarily, the pan-tilt attitude information may include the pitch angle, yaw angle, and roll angle of the pan-tilt to indicate the specific direction of the pan-tilt of the vehicle-mounted mobile imaging device in three-dimensional space. The pan-tilt camera shooting parameters may include focal length, field of view, exposure settings, etc., so as to guide the quality or style of the images captured by the pan-tilt. The pan-tilt position information may include the actual position of the pan-tilt of the vehicle-mounted mobile imaging device in space, such as the position adjustment in the horizontal or vertical direction controlled by a telescopic rod, so as to ensure that the pan-tilt can be in the desired shooting position.
[0030] In an example of the embodiment of the present application, the target pan-tilt perspective orientation information corresponding to the target pan-tilt perspective identifier is parsed through an identification decoding algorithm. Exemplarily, according to the format of the identifier, the identification decoding algorithm parses the target pan-tilt perspective identifier into corresponding pan-tilt control parameters, and can extract the bit fields corresponding to each control dimension from the pan-tilt perspective identifier according to a predetermined coding format. For example, the first 3 bits of the target pan-tilt perspective identifier may correspond to the pitch angle of the pan-tilt.
[0031] In another example of the embodiment of the present application, the target pan-tilt orientation information matching the target pan-tilt perspective identifier is searched from a library table, so that the pan-tilt perspective orientation corresponding to the target perspective identifier can be accurately determined. Exemplarily, a plurality of relationships between the pan-tilt perspective identifier and the pan-tilt perspective orientation information are pre-stored in the library table, and the corresponding target pan-tilt perspective orientation information can be quickly located by searching with the target pan-tilt perspective identifier.
[0032] Thus, by accurately matching the target perspective identifier with the orientation information of the pan-tilt, highly accurate perspective adjustment can be achieved, ensuring the accuracy and consistency of the shooting angle requested by the user each time.
[0033] In step S130, the pan-tilt perspective of the vehicle-mounted mobile imaging device is controlled and adjusted according to the target pan-tilt perspective orientation parameter.
[0034] In some embodiments, a control signal is sent to the vehicle-mounted mobile imaging device or the vehicle-mounted bracket to adjust the pan-tilt attitude, pan-tilt camera shooting parameters, or pan-tilt position of the vehicle-mounted mobile imaging device according to the target pan-tilt perspective orientation parameter in the control signal, so as to ensure that the pan-tilt perspective matches the target pan-tilt perspective orientation parameter.
[0035] Through the embodiments of the present application, by automatically regulating the instructions for the pan-tilt of the vehicle-mounted mobile imaging device, the driving risk caused by manual operation of the vehicle-mounted mobile imaging device is avoided, and the shooting angles after each adjustment can be ensured to be accurate and consistent. The pan-tilt can quickly adapt and adjust to the perspective expected by the user through instruction regulation, ensuring the consistency of the shooting screen, thereby meeting the best perspective shooting effect expected by the user.
[0036] Figure 2 The operation flowchart showing an example of obtaining a perspective control request according to an embodiment of the present application is shown.
[0037] As Figure 2 shown, in step S210, user input information is obtained, and it is detected whether the user input information includes a pan-tilt perspective identifier.
[0038] Here, the user input information includes any one of the following: graphical interface input information, voice input information, gesture input information, button input information, or joystick input information.
[0039] Exemplarily, in the graphical interface input information, the user clicks or drags through a touch screen or a graphical interface to select a specific perspective, for example, operates through interactive elements such as buttons, drop-down menus, sliders, etc. In the voice input information, the user controls the pan-tilt through voice commands, for example, the user says "adjust to the left front side perspective". In the gesture input information, the user adjusts the perspective through gesture control. For example, a waving gesture or a specific finger movement indicates the moving direction of the pan-tilt. In the button input information, the user can operate through physical buttons (such as buttons in vehicle-mounted devices), and the buttons may be bound to specific perspective or action instructions. In the joystick input information, the user can control the perspective change of the pan-tilt through joystick or lever input.
[0040] Thus, the system supports multiple input methods, enabling the user to flexibly select the most suitable control method for themselves (such as graphical interface, voice, gesture, etc.), enabling the system to efficiently parse the user input and identify the pan-tilt perspective identifier, improving the intuitiveness and convenience of user operation.
[0041] In step S220, when it is detected that the user input information includes a pan-tilt perspective identifier, a perspective control request is generated according to the pan-tilt perspective identifier included in the user input information.
[0042] In some embodiments, when it is detected that the input information of the user contains a pan-tilt perspective identifier, the system will extract the identifier from the input information. For example, the system obtains a corresponding perspective identifier (such as "front perspective" or "front left side perspective") through speech recognition or graphical interface element parsing. Furthermore, based on the extracted pan-tilt perspective identifier, the system will generate a perspective control request and send it to the pan-tilt control module of the in-vehicle mobile imaging device or other relevant hardware (such as an in-vehicle bracket) for actual execution, controlling and adjusting the posture, position or shooting parameters of the pan-tilt, and ensuring that the pan-tilt can be accurately adjusted according to the user's needs.
[0043] Figure 3 The schematic diagram of the interface effect shows an example of the graphical interaction interface corresponding to the graphical interface input information.
[0044] Such as Figure 3 , which shows an example of the graphical interface in the in-vehicle mode. The left side of the graphical interface is the interaction area, which is convenient for the user to adjust the pan-tilt perspective azimuth parameters of the in-vehicle mobile imaging device by initiating interaction commands. The right side of the graphical interface is the perspective display area, which is used for the user to view the images captured by the in-vehicle mobile imaging device under the corresponding pan-tilt perspective azimuth parameters.
[0045] In some examples of the embodiments of the present application, based on at least one perspective option in the graphical interaction interface, the user trigger operation is detected, and each perspective option is respectively configured with a corresponding pan-tilt perspective identifier.
[0046] Referring to the example of the interaction area such as Figure 3 , on the vehicle model in the interaction area, a plurality of interaction points (such as the driver's seat, passenger seat, rear seat, dead ahead, etc.) are preset, and each point is bound to a specific pan-tilt perspective identifier. Each interaction point supports click or touch operations, which is convenient for the user to quickly select the target perspective. In addition, Figure 3 a virtual joystick is also provided in the interaction area shown, which can also be used by the user to further finely control the set angle indicated by the pan-tilt perspective identifier to meet the more refined pan-tilt perspective control requirements.
[0047] Furthermore, when the user trigger operation for the target perspective option is detected, a perspective control request is generated according to the pan-tilt perspective identifier corresponding to the target perspective option.
[0048] In some embodiments, the system pre-configures the mapping relationship between each perspective identifier and specific pan-tilt posture parameters (pitch angle, yaw angle, etc.). For example, the "driver's seat" perspective may correspond to "yaw: +10°, pitch: -5°". Once the user selects this option, these parameters can be automatically called to control the pan-tilt.
[0049] Through the embodiments of the present application, a fast, intuitive, and intelligent pan-tilt perspective control process is realized by means of "graphical options + parameter mapping + instant feedback", enabling users to complete high-precision perspective switching of in-vehicle mobile imaging devices without understanding complex angle control principles. Simply by clicking on the interface options, complex pan-tilt control actions are abstracted into one-key selections, reducing the operation threshold for users. This allows users to intuitively and conveniently trigger command requests through active operations, achieve precise control of the pan-tilt perspective, and ensure the consistency of pan-tilt angle adjustment results.
[0050] It should be noted that during vehicle driving, in-vehicle mobile imaging devices are often used to capture beautiful scenery encountered on the way. However, in an in-vehicle scenario, the position of the handheld camera depends on the position where the user fixes it with a bracket, and during driving, the process of reaching out to use a joystick to control the pan-tilt is relatively complex, affecting driving safety. As a result, during the shooting process, the perspective of the images captured by the handheld camera is often very single, and the operation of switching perspectives is also rather troublesome.
[0051] In the business application scenario combining the embodiments of the present application, perspective control requests can be triggered by various multi-modal user input information, thereby automatically completing the adjustment of the pan-tilt perspective of the in-vehicle mobile imaging device. This enables quick and convenient selection of specific perspectives without reaching out to operate the physical joystick on the device to adjust the pan-tilt angle, and ensures the consistency of the shooting perspective.
[0052] In some examples of the embodiments of the present application, in addition to users actively triggering pan-tilt perspective control operations, the system can also support automatic triggering of pan-tilt perspective control in various specific scenarios. Without the need for users to actively operate, it can automatically capture high-value video segments, such as fleeting scenery, moments when passengers in the vehicle are laughing, and so on.
[0053] Figure 4 Shows an operation flowchart of an example for obtaining a perspective control request according to an embodiment of the present application.
[0054] As Figure 4 shown, in step S410, obtain the emotion state detection results of at least one passenger in the vehicle.
[0055] Here, various known or potential passenger emotion detection systems can be used to monitor the emotion state of passengers in the vehicle. Exemplarily, install a facial recognition camera in the vehicle to collect the physiological and facial expression data of passengers in the vehicle in real time. Analyze the facial expressions, eye movements, smiles, frowns, etc. of passengers through facial recognition, and combine machine learning models to judge the emotion state of passengers (such as happy, surprised, relaxed, annoyed, etc.).
[0056] In step S420, when the emotion state detection result of the first in-vehicle occupant meets the preset emotion condition, determine the first pan-tilt perspective identifier that matches the seating position of the first in-vehicle occupant.
[0057] In some embodiments, the system presets the trigger conditions for different emotion states. For example, when the system detects that the emotion score of a certain occupant in the vehicle exceeds a certain threshold and is a positive emotion such as happiness, smile, surprise, etc., a request for pan-tilt control is triggered. For example, when it is detected that the occupant emits laughter and the facial expression shows joy, the system will recognize this as a moment worth capturing. Furthermore, the pan-tilt perspective is determined by combining the emotion state and the specific position of the occupant (front row, rear row, left side or right side, etc.). Referring to Figure 3 the example in, when it is detected that the occupant in the front row is laughing particularly happily, the system will select a pan-tilt perspective suitable for photographing the occupant in the front row to record the happy moment of the occupant in the front row.
[0058] In step S430, generate a perspective control request based on the first pan-tilt perspective identifier.
[0059] In some embodiments, based on the matching of the emotion state detection result and the occupant position, the system generates a corresponding perspective control request to send a specific pan-tilt adjustment command to the in-vehicle mobile imaging device, such as the pitch angle, yaw angle, roll angle, focal length, exposure parameter, etc. of the pan-tilt. Thus, the pan-tilt adjustment operation is automatically triggered and executed, and the entire process does not require user intervention, ensuring that the pan-tilt can be quickly adjusted to the best shooting position at the moment when the emotion change is captured.
[0060] Through the embodiments of the present application, when identifying and capturing wonderful moments such as laughter and surprise moments during the emotion change of the occupant, the system automatically adjusts the pan-tilt perspective according to the emotion state and the occupant position without the need for the user to actively operate. Thus, based on the emotion state change and seating position of the in-vehicle occupant, the pan-tilt perspective is automatically adjusted to capture high-value in-vehicle occupant image segments, enhancing the intelligence and user-friendliness of the in-vehicle imaging device.
[0061] Figure 5 The operation flowchart showing an example of obtaining a perspective control request according to an embodiment of the present application is shown.
[0062] As Figure 5 shown, in step S510, obtain in-vehicle map information and detect whether there is a user interest point near the vehicle according to the in-vehicle map information, or, obtain the vehicle environment information captured by the in-vehicle camera and detect whether there is a user interest point near the vehicle according to the vehicle environment information.
[0063] It should be noted that a Point of Interest (POI) refers to a specific location or object related to the user's current needs, preferences, or behavior patterns. It can be a specific geographical location or a particular target in the environment. Exemplarily, POIs can include map-based POIs, such as restaurants or parks marked in a high-precision map database. In addition, POIs can also include environment-based POIs, which can generally be detected in real time through in-vehicle cameras or other sensors, such as beautiful weather scenes that match the user's preferences, roadside billboards, etc.
[0064] In some embodiments, in-vehicle map information is obtained in real time through an in-vehicle navigation system, including the vehicle's current geographical location, driving route, surrounding landmarks, etc., and nearby POI information is extracted by combining with a high-precision map database. Furthermore, according to the preferences set by the user (such as "display the nearest restaurant") or the default options recommended by the system, eligible POIs are filtered out, and by calculating the distance between the vehicle and the POI (such as setting a threshold range, for example, within 500 meters), it is determined whether there are eligible POIs.
[0065] On the other hand, the in-vehicle camera is used to collect the environmental images around the vehicle in real time, including the buildings and weather scenes on both sides of the road, and computer vision algorithms (such as object detection, semantic segmentation, etc.) can be used to process the images to identify POIs in the environmental information, such as identifying sunsets, starry skies, or rainbows through the weather scenes in the environmental information.
[0066] Thus, by integrating in-vehicle map information and camera environmental data, the system can intelligently identify and detect potential POIs near the vehicle to facilitate capturing high-value images in the surrounding environment outside the vehicle.
[0067] In step S520, when there is a user POI, a matching pan-tilt view angle identifier is determined according to the relative orientation between the vehicle and the user POI.
[0068] In some embodiments, the in-vehicle system obtains the vehicle's current position and orientation angle (heading angle) through the GPS module, calculates the relative orientation between the vehicle and the POI according to the geographical location (latitude and longitude coordinates) of the POI (such as in front, left, right, behind, etc.), and the relative orientation can also be expressed in angles, such as the angular offset of the POI relative to the vehicle's due front.
[0069] Furthermore, the system generates a pan-tilt view angle identifier that matches this position and angle, which can include or be used to indicate the specific angles (such as pitch angle, yaw angle, etc.) to which the pan-tilt of the in-vehicle mobile imaging device needs to be adjusted, as well as camera settings (such as focal length, exposure, etc.).
[0070] In step S530, a view control request is generated based on the second pan-tilt view identifier.
[0071] Here, according to the calculated second pan-tilt view identifier, the system generates a pan-tilt control command to guide the pan-tilt of the in-vehicle mobile imaging device to be adjusted to the corresponding position, attitude, and focal length. In this way, after the adjustment, the pan-tilt will automatically point to the point of interest and perform automatic shooting without the intervention of the user, automatically completing the shooting of the user's point of interest.
[0072] Through the embodiments of the present application, by combining the in-vehicle map information with the environmental information captured by the in-vehicle camera, it is possible to automatically detect and identify the user's points of interest near the vehicle, and trigger the automatic adjustment of the view of the pan-tilt based on the relative orientation between the vehicle and the point of interest. It can automatically identify the user's points of interest according to the in-vehicle map and environmental information and automatically adjust the view of the pan-tilt, achieving a better shooting view of the target point of interest and ensuring that the best view of the user's point of interest can be captured.
[0073] In particular, in the dynamic changing environment scenario of vehicle driving, it is possible to automatically trigger the best view adjustment for sudden or fleeting user points of interest without the user's active trigger, and it can also ensure that the captured images are of high quality and meet the user's needs.
[0074] Figure 6 Fig. shows a schematic diagram of the shooting effect of an example for the user's point of interest according to the embodiments of the present application.
[0075] As Figure 6 shown, it shows the shooting effect of the in-vehicle mobile imaging device on the user's point of interest of sunset. During the vehicle driving process, the environmental images around the vehicle are collected in real time through the in-vehicle camera, and it is recognized that the weather scenery contains the user's point of interest, and the pan-tilt is adjusted for shooting according to the pan-tilt view identifier of the front view, so as to automatically identify the high-value scenes (such as sunset, etc.) around and adjust the matching best pan-tilt view, achieving high-quality image capture of the high-value scenes encountered in the vehicle driving environment.
[0076] In some examples of the embodiments of the present application, the pan-tilt view correspondence defines the association relationship between the pan-tilt view identifier and the pan-tilt view azimuth parameter. For example, different pan-tilt view identifiers (such as "front view", "left view", "top view", etc.) are respectively uniquely mapped to the corresponding pan-tilt view azimuth parameters. In this way, when the system receives the view control request, it can quickly and accurately determine the target pan-tilt view azimuth information matching the target pan-tilt view identifier according to the pan-tilt view correspondence, and convert it into a specific pan-tilt control instruction, so as to achieve accurate pan-tilt view adjustment.
[0077] It should be understood that, on the one hand, the corresponding relationship between the pan-tilt angles can be set by the system default or recommendation. For example, multiple pan-tilt angle identifiers and corresponding pan-tilt angle orientation parameters are default set in the system according to experimental tests. On the other hand, the corresponding relationship between the pan-tilt angles can also be set or adjusted according to user needs. For example, users can adopt specific pan-tilt angle identifiers or special pan-tilt angle orientation parameters to meet their personalized vehicle-mounted image acquisition needs.
[0078] Figure 7 Fig. shows an operation flowchart of an example of setting the corresponding relationship between the pan-tilt angles according to an embodiment of the present application.
[0079] As Figure 7 shown, in step S710, a pan-tilt angle entry request is obtained, and the pan-tilt angle entry identifier corresponding to the pan-tilt angle entry request and the pan-tilt angle orientation entry parameters of the vehicle-mounted mobile imaging device are parsed.
[0080] It should be understood that the pan-tilt angle entry request can adopt the same or similar acquisition method as the viewing angle control request, and can also be diversified or multimodal. For example, the pan-tilt angle entry request can be initiated through a vehicle-mounted display screen, a mobile application APP, or a vehicle-mounted voice assistant, etc.
[0081] Specifically, the pan-tilt angle entry request can include a new pan-tilt angle identifier that the user hopes to define, and the user can choose to adjust the existing viewing angle or create a completely customized viewing angle as the corresponding pan-tilt angle orientation parameter, and different pan-tilt angles are bound and distinguished by the viewing angle identifier. In this way, users can freely set new pan-tilt angle identifiers according to their own needs and create customized viewing angles related to personal needs or specific scenarios.
[0082] Combined with the example of the interaction interface as Figure 3 shown, the user can enter the viewing angle editing state by selecting or adding points (or camera positions) on the vehicle model, so as to realize the editing or addition of the corresponding relationship between the pan-tilt angles. In addition, each point in the vehicle model can be adjusted or freely placed by the user according to their preferences to meet the user's personalized visual experience requirements for the control interface, and it is also convenient for the user's subsequent quick operation and call. It should also be noted that the layout of the points in the vehicle model does not necessarily indicate the corresponding azimuth viewing angle. For example, the point of "front left" can correspond to the pan-tilt angle orientation of "front left" or the pan-tilt angle orientation of "rear left", and it can be customarily bound and set according to the user's preferences.
[0083] In step S720, the corresponding relationship between the pan-tilt angles is set according to the pan-tilt angle entry identifier and the pan-tilt angle orientation entry parameters.
[0084] Here, the identifier in the pan-tilt view entry request is associated with the orientation parameter through the pan-tilt view correspondence relationship, ensuring that when a matching view control request is received subsequently, the identifier can be accurately and quickly called and converted into specific pan-tilt control parameters.
[0085] It should be understood that the storage method and expression form of the pan-tilt view correspondence relationship can be diversified, and different solutions can be selected according to system requirements and technical architectures. For example, structured data storage (such as database tables), semi-structured data storage (such as JSON files, XML files), or unstructured data storage (such as key-value pair storage) can be adopted. In addition, the pan-tilt view correspondence relationship can also be dynamically managed through in-memory data structures (such as dictionaries, hash tables) in programming languages, and all fall within the scope of implementation of the embodiments of the present application.
[0086] Through the embodiments of the present application, users can flexibly set the pan-tilt view according to their own needs, save it as a custom view identifier, and set clear view orientation parameters for each pan-tilt view identifier, enabling the system to achieve fast and accurate pan-tilt control, avoiding the need for users to manually adjust the view each time, and improving the refinement level of automated pan-tilt control and the personalized pan-tilt view shooting experience.
[0087] Regarding the acquisition of the pan-tilt view entry request in step S710, in some embodiments, when detecting user manual control information for the pan-tilt view orientation parameter of the vehicle-mounted mobile imaging device, a pan-tilt view save notification is generated.
[0088] Exemplarily, when the user adjusts the pan-tilt view through a vehicle-mounted control interface (such as a vehicle-mounted touch screen, physical knob, voice control, etc.), the system will detect and record the user's manual control information in real time, such as adjusting the pitch angle, yaw angle, and roll angle of the pan-tilt. Combining with the example of the interaction interface in Figure 3 when detecting that the user adjusts the pan-tilt view position through a virtual joystick, a pan-tilt view save notification is generated.
[0089] At this time, a pan-tilt view save notification will be generated, which is used to ask the user whether to save the new pan-tilt view formed by this manual control. Preferably, more specific control information can also be recorded in the notification content, such as the pan-tilt orientation parameters (including all adjusted angle values and camera settings) manually adjusted by the user, and can be presented to the user in the form of a prompt box, pop-up window, or voice feedback.
[0090] Furthermore, in the case of detecting the user's confirmation feedback for the pan-tilt view save notification, a new pan-tilt view identifier is generated, and a pan-tilt view entry request is generated in combination with the pan-tilt view orientation parameters adjusted and updated based on the user's manual control information.
[0091] In some embodiments, the pan-tilt view saving notification is a prompt box interface, which displays options such as "Confirm" or "Cancel" for the user to operate in response to the system's request. If the user selects Confirm, the system will continue to process and generate a new pan-tilt view identifier; if Cancel is selected, the system will abandon the saving setting for the current manually adjusted view.
[0092] Through the embodiments of the present application, when it is detected that the user manually adjusts the pan-tilt view azimuth parameter, the user is asked whether to save the custom view through a confirmation feedback mechanism, and a corresponding pan-tilt view identifier is automatically generated, realizing flexible recording of the user's personalized configuration of the pan-tilt view, enabling the user to easily create and save custom views in different scenarios, meeting the user's need to capture different scenarios or details in real time during driving, and enhancing the adaptability and flexibility of the view adjustment of the in-vehicle imaging device.
[0093] Figure 8 Shows an interface interaction change diagram of an example of view operation on a graphical interface according to an embodiment of the present application.
[0094] As Figure 8 shown, by operating the graphical interface of the mobile application APP or the in-vehicle terminal, the view that the user wants to save is recorded, allowing the user to quickly select the desired view subsequently.
[0095] First, enter the view recording function. The user can operate the joystick on the device / the virtual joystick in the APP, adjust to the desired view and click Save, and the system will automatically record various parameters of the current view. The user can create multiple views at the same time and can operate by using a physical joystick (if equipped) or a virtual joystick on the touch screen. The adjusted content includes the pitch angle (up and down), yaw angle (left and right) of the pan-tilt, and possibly the focal length or other camera settings. During the adjustment process, the operation interface updates the captured image in real time, enabling the user to intuitively see the change of the current view and ensuring the accuracy and timeliness of the view adjustment.
[0096] After exiting the view recording function, the user can click the previously set view button on the interface, and the device will automatically turn to the selected view. Exemplarily, a series of saved view buttons can be seen on the main interface, and each view button corresponds to a view identifier saved by the user. The user only needs to click the desired view button, and the device will automatically adjust the pan-tilt to the corresponding angle and switch to the saved view. Thus, through one-key switching, quickly switch to the saved view, avoiding cumbersome manual adjustment, improving the operation efficiency, and greatly enhancing the operation convenience of the user.
[0097] In some preferred embodiments, when a high-value scenario is automatically recognized, the pan-tilt head can also be adjusted by calling the corresponding perspective parameters to help the user record high-quality video content of interest to the user.
[0098] Further, after the user selects a perspective, if the user continues to operate the device's virtual joystick, the interactive interface will prompt the user whether to save the current perspective parameters or whether to add a new perspective. Saving the current perspective will update the setting parameters of the selected perspective, and adding a new perspective will add an option for a new perspective. Thus, the system can flexibly manage multiple perspective configurations, and the user can quickly update the existing perspective or create a new perspective as needed without manual adjustment multiple times.
[0099] Through the embodiments of the present application, the user can preset multiple perspectives and can quickly switch between them with one click during subsequent use, saving the time and effort of manual adjustment and making shooting more personalized and flexible. In addition, by switching the saved perspectives with one click, the user can quickly enter the required shooting scene, greatly improving the operation efficiency and convenience, and significantly enhancing the intelligent level and user operation experience of the in-vehicle imaging device.
[0100] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of actions combined. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application. In the above embodiments, each embodiment is described with emphasis. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0101] In some embodiments, the embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer is enabled to execute any one of the above-mentioned control methods for a mobile imaging device in an in-vehicle scenario.
[0102] In some embodiments, the embodiments of the present application also provide an electronic device, which includes: at least one processor, and a memory communicatively connected to the at least one processor. Wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the control method for a mobile imaging device in an in-vehicle scenario.
[0103] The device in the embodiment of the present application described above can be used to execute the method for controlling a mobile imaging device for in-vehicle scenarios in the embodiment of the present application, and correspondingly achieve the technical effects achieved by the method for controlling a mobile imaging device for in-vehicle scenarios in the embodiment of the present application, which will not be elaborated here. In the embodiment of the present application, relevant functional modules can be implemented by a hardware processor.
[0104] Figure 9 FIG. is a schematic hardware structure diagram of an electronic device for executing the method for controlling a mobile imaging device for in-vehicle scenarios provided by another embodiment of the present application. As Figure 9 shown, the device includes: One or more processors 910 and a memory 920. Figure 9 Here, one processor 910 is taken as an example.
[0105] The device for executing the method for controlling a mobile imaging device for in-vehicle scenarios may further include: an input device 930 and an output device 940.
[0106] The processor 910, the memory 920, the input device 930, and the output device 940 may be connected through a bus or other means. Figure 9 Here, the connection through a bus is taken as an example.
[0107] As a non-volatile computer-readable storage medium, the memory 920 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions / modules corresponding to the method for controlling a mobile imaging device for in-vehicle scenarios in the embodiment of the present application. By running the non-volatile software programs, instructions, and modules stored in the memory 920, the processor 910 executes various functional applications and data processing of the server, that is, implements the method for controlling a mobile imaging device for in-vehicle scenarios in the above method embodiment.
[0108] The memory 920 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the device. In addition, the memory 920 may include a high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 920 may optionally include a memory remotely provided with respect to the processor 910, and these remote memories may be connected to the device through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.
[0109] The input device 930 can receive input digital or character information and generate signals related to the user settings and function controls of the device. The output device 940 can include display devices such as a display screen.
[0110] The one or more modules are stored in the memory 920 and, when executed by the one or more processors 910, perform the method for controlling a mobile imaging device for an in-vehicle scenario in any of the above method embodiments.
[0111] The above product can execute the method provided in the embodiments of the present application, and has the corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference can be made to the method provided in the embodiments of the present application.
[0112] The electronic devices in the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication devices: Such devices are characterized by having mobile communication functions and mainly aim to provide voice and data communication. Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones, etc.
[0113] (2) Ultra-mobile personal computer devices: Such devices belong to the category of personal computers, have computing and processing functions, and generally also have the characteristic of mobile Internet access. Such terminals include: PDAs, MIDs, and UMPC devices, etc., such as iPad.
[0114] (3) Portable entertainment devices: Such devices can display and play multimedia content. Such devices include: audio and video players (such as iPod), handheld game consoles, e-books, and smart toys and portable in-vehicle navigation devices.
[0115] (4) Servers: Devices that provide computing services. The composition of a server includes a processor, a hard disk, a memory, a system bus, etc. Servers are similar to general computer architectures, but due to the need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, manageability, etc.
[0116] (5) Other electronic devices with data interaction functions.
[0117] In some embodiments, the present application also provides a mobile platform, which installs the computer device described in any of the embodiments of the present application. The mobile platform includes but is not limited to vehicles, tracked robots, biped robots, quadruped robots, etc., where the vehicle can be a passenger car, a pickup truck, a freight truck, etc. It should be noted that the above are only examples, and the present application does not limit the specific form of the mobile platform.
[0118] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0119] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the related technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for controlling a mobile imaging device for vehicle-mounted scenarios, comprising: Obtaining a perspective control request and parsing a target pan-tilt perspective identifier corresponding to the perspective control request; Determining target pan-tilt perspective orientation information that matches the target pan-tilt perspective identifier; the parameter type of the pan-tilt perspective orientation parameter includes at least one of the following: pan-tilt attitude information, pan-tilt camera shooting parameters, or pan-tilt position information; Controlling and adjusting the pan-tilt perspective of the vehicle-mounted mobile imaging device according to the target pan-tilt perspective parameter; the vehicle-mounted mobile imaging device is a mobile imaging device in a vehicle-mounted scenario.
2. The method according to claim 1, wherein The obtaining of the perspective control request includes: Obtaining user input information and detecting whether the user input information includes a pan-tilt perspective identifier; the user input information includes any one of the following: graphical interface input information, voice input information, gesture input information, button input information, or joystick input information; When it is detected that the user input information includes a pan-tilt perspective identifier, generating a perspective control request according to the pan-tilt perspective identifier included in the user input information.
3. The method according to claim 2, wherein The obtaining of the user input information and detecting whether the user input information includes a pan-tilt perspective identifier includes: Detecting a user trigger operation based on at least one perspective option in the graphical interaction interface; each of the perspective options is configured with a corresponding pan-tilt perspective identifier; The when it is detected that the user input information includes a pan-tilt perspective identifier, generating a perspective control request according to the pan-tilt perspective identifier included in the user input information includes: When it is detected a user trigger operation for a target perspective option, generating a perspective control request according to the pan-tilt perspective identifier corresponding to the target perspective option.
4. The method according to claim 1, wherein The obtaining of the perspective control request includes: Obtaining the emotion state detection results of at least one vehicle occupant; When the emotion state detection result of the first vehicle occupant meets a preset emotion condition, determining a first pan-tilt perspective identifier that matches the seating position of the first vehicle occupant; Generating a perspective control request based on the first pan-tilt perspective identifier.
5. The method according to claim 1, wherein, The obtaining of the perspective control request includes: Obtaining vehicle-mounted map information and detecting whether there is a user interest point near the vehicle according to the vehicle-mounted map information, or obtaining the vehicle environment information captured by the vehicle-mounted camera and detecting whether there is a user interest point near the vehicle according to the vehicle environment information; When there is the user interest point, determining a matching pan-tilt perspective identifier according to the relative orientation between the vehicle and the user interest point; Generating a perspective control request based on the second pan-tilt perspective identifier.
6. The method according to any one of claims 1-5, wherein, The determining of the target pan-tilt perspective orientation information that matches the target pan-tilt perspective identifier includes: Determining the target pan-tilt perspective orientation information that matches the target pan-tilt perspective identifier according to the pan-tilt perspective correspondence; the pan-tilt perspective correspondence defines the association relationship between the pan-tilt perspective identifier and the pan-tilt perspective orientation parameter; Regarding the setting of the pan-tilt perspective correspondence, it includes: Obtaining a pan-tilt perspective entry request and parsing the pan-tilt perspective entry identifier and the pan-tilt perspective orientation entry parameter of the vehicle-mounted mobile imaging device corresponding to the pan-tilt perspective entry request; Set the corresponding relationship of the pan-tilt angle according to the pan-tilt angle entry identifier and the pan-tilt angle orientation entry parameter.
7. The method according to claim 6, wherein, The obtaining of the pan-tilt angle entry request includes: When detecting user manual control information for the pan-tilt angle orientation parameter of the in-vehicle mobile imaging device, generate a pan-tilt angle save notification; In the case of detecting a confirmation feedback from the user for the pan-tilt angle save notification, generate a new pan-tilt angle identifier, and generate the pan-tilt angle entry request in combination with the pan-tilt angle orientation parameter that is adjusted and updated based on the user manual control information.
8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, wherein, The processor executes the computer program to implement the steps of the method according to any one of claims 1-7.
9. A storage medium, in which one or more programs including execution instructions are stored, and the execution instructions can be read and executed by an electronic device to be used for implementing the steps of the method according to any one of claims 1-7.
10. A computer program product, the computer program product includes a computer program stored on a storage medium, the computer program includes program instructions, when the program instructions are executed by a computer, the computer is caused to implement the steps of the method according to any one of claims 1-7.