Mobile communication device and method for providing user guidance information of vehicle components

Abstract

This application provides a mobile communication device. The mobile communication device includes an imaging module, a control module, and a display module. The imaging module is configured to capture images of at least a portion of vehicle components. The control module includes a storage unit, a processing unit, and a recognition unit. The storage unit stores reference images of multiple vehicle components and user guidance information for multiple vehicle components. The processing unit is configured to adjust the viewing angle orientation of the reference images. The recognition unit is configured to determine the components in the captured image based on the adjusted reference images. The display module is configured to display the user guidance information for the determined components in response to an instruction from the control module. This application also provides a method for outputting user guidance information for vehicle components.

Description

[Technical field]

[0001] The present application relates to a mobile communication device and method for providing user guidance information of vehicle components, and in particular to a mobile communication device and method capable of identifying vehicle components according to images captured by an imaging module and outputting user guidance information of the vehicle components. [Background technology]

[0002] A vehicle includes a variety of components, and some components (e.g., operating buttons) are similar in appearance but have different functions. Users can use paper user guides provided by automobile manufacturers to find user guidance information for components. For some users, it is time-consuming to accurately find the user guidance information for the component. Currently, there are applications that use images captured by cameras of portable communication devices to identify components and provide user guidance information. However, the capture angle and capture environment (e.g., light, etc.) of the component may affect the communication device's recognition of the component. Therefore, there is a need for a device that can simply and quickly identify vehicle components and output user guidance information related thereto. [Summary of the invention]

[0003] In response to at least one of the above-mentioned problems, the present application provides a mobile device that can simply and quickly identify vehicle parts and output user guidance information. It does not need to process or store a large number of reference pictures, can recommend possible parts for display, and can guide users to obtain easily recognizable pictures to achieve accurate identification.

[0004] According to one aspect of the present application, a mobile communication device is provided. The mobile communication device includes an imaging module, a control module, and a display module. The imaging module is configured to capture at least a partial image of a component of a vehicle. The control module includes a storage unit, a processing unit, and a recognition unit. The storage unit stores reference images of multiple vehicle components and user guidance information of multiple vehicle components. The processing unit is configured to adjust the viewing angle orientation of the reference image. The recognition unit is configured to determine the component in the captured image based on the adjusted reference image. The display module is configured to display the user guidance information of the determined component in response to the instruction of the control module.

[0005] In some embodiments, the reference image in the storage unit is provided by a three-dimensional stereo model of the virtual vehicle stored in the storage unit.

[0006] In some embodiments, the processing unit is configured to adjust the orientation of the three-dimensional solid model based on the orientation of the imaging module relative to the captured component so as to provide a reference image that is substantially consistent with the captured image.

[0007] In some embodiments, the orientation of the imaging module relative to the captured component is based on an angle between a line connecting the imaging module and the captured component and an X-axis, a Y-axis, or a Z-axis of the vehicle.

[0008] In some embodiments, the processing unit is configured to randomly adjust the orientation of the three-dimensional stereo model until the three-dimensional stereo model provides a reference image that is substantially consistent with the captured image.

[0009] In some embodiments, the control module is further configured to: instruct the display module to display user guide information of the components associated with the determined component.

[0010] In some embodiments, the associated components are components that are related to the determined component locations.

[0011] In some embodiments, the associated components are components that are related to the determined component functionality.

[0012] In some embodiments, the control module is further configured to: instruct the display module to display user guidance information of a plurality of components that have high similarity to the components in the captured image.

[0013] In some embodiments, the mobile communication device further comprises an image detection module configured to detect contrast and / or sharpness of a captured image.

[0014] In some embodiments, the control module is further configured to: when the image detection module determines that the contrast of the captured image is not within a threshold range, instruct the display module to output a prompt to adjust the ambient brightness to capture the image again.

[0015] In some embodiments, the control module is further configured to: when the image detection module determines that the clarity of the captured image is not within a threshold range, instruct the display module to output a prompt to refocus to capture the image.

[0016] According to another aspect of the present application, a method for outputting user guidance information of a vehicle component is provided. The method includes: receiving a captured image of a vehicle component; adjusting the viewing angle orientation of a plurality of reference images of stored vehicle components; identifying the vehicle component in the captured image based on the adjusted reference images of the plurality of vehicle components; and outputting user guidance information of the identified vehicle component.

[0017] In some embodiments, the method also includes: providing a reference image through a virtual three-dimensional stereo model of the vehicle; determining the orientation of the capture action relative to the vehicle component; and adjusting the orientation of the three-dimensional stereo model based on the orientation of the imaging device used for capture relative to the vehicle component so that it provides a reference image that is substantially consistent with the captured image.

[0018] In some embodiments, the orientation of the imaging device relative to the vehicle component is based on an angle between a line connecting the imaging device and the vehicle component and an X-axis, a Y-axis, or a Z-axis of the vehicle.

[0019] In some embodiments, the method further includes outputting user guidance information of a plurality of vehicle components associated with the identified component or a plurality of vehicle components having high similarity to the component of the captured image.

[0020] In some embodiments, the method further comprises: detecting contrast and / or sharpness of the captured image.

[0021] In some embodiments, the method further includes instructing the user to adjust ambient brightness and re-capturing the image of the vehicle component when the contrast of the captured image is not within a threshold range.

[0022] In some embodiments, the method further includes instructing the user to refocus and capture the image of the vehicle component again when the clarity of the captured image is not within a threshold range.

[0023] In some embodiments, outputting includes displaying an image of the identified vehicle component together with user guidance information associated therewith.

[0024] It should be understood that the above brief description is provided to introduce a series of selected concepts that will be further described in the detailed description in a simplified form, and it is not meant to identify the key or essential features of the protected subject matter of the present invention, and the scope of the present invention will be uniquely defined by the claims of the present invention. In addition, the protected subject matter is not limited to embodiments that overcome any disadvantages described above or in any part of this specification.

Brief Description of the Drawings

[0025] One or more features and / or advantages of the present invention will become apparent from one or more embodiments described in detail below in conjunction with the accompanying drawings.

[0026] Figure 1 is a block diagram of a mobile communication device according to an embodiment of the present application, showing various sub-units of the mobile communication device.

[0027] Figure 2A and Figure 2B yes Figure 1 Schematic diagram of a mobile communication device in the figure, showing the image captured by the imaging module and the information output by the display device respectively.

[0028] Figure 3A and 3B yes Figure 1 Schematic diagram of a mobile communication device in FIG. 1 , showing component information associated with an identified component and component information having a high degree of similarity to the identified component.

[0029] Figure 4A is a schematic diagram showing another vehicle component captured by a mobile communication device according to another embodiment of the present application, Figure 4B Shows Figure 4AThe mobile communication device recognizes two states of the virtual three-dimensional model of the vehicle in the control module. Figure 4C and Figure 4D They are Figure 4B An enlarged schematic diagram of the adjustment state of the three-dimensional model in the original state.

[0030] Figure 5 The present invention is a flowchart of a method for outputting user guide information of a vehicle component according to an embodiment of the present application. [Specific implementation method]

[0031] As required, specific embodiments of the present invention are disclosed in the present application specification; however, it should be understood that the embodiments disclosed herein are merely examples of the present invention that can be implemented in multiple, alternative forms. The drawings need not be drawn to scale; some features may be enlarged or reduced to show the details of specific components. The same or similar reference numerals may indicate the same parameters and components or similar modifications and alternatives thereto. In the description below, multiple operating parameters and components are described in multiple contemplated embodiments. These specific parameters and components are used in this specification only as examples and are not meant to be limiting. Therefore, the specific structural and functional details disclosed in this specification should not be understood as limitations, but merely as a representative basis for teaching those skilled in the art to implement the present invention in multiple forms.

[0032] Figure 1 A block diagram of a mobile communication device 100 for providing user guidance information according to an embodiment of the present application is shown, showing the various sub-units therein. The mobile communication device 100 for providing user guidance information may include a mobile device with a camera and a control module. For example, the mobile communication device 100 may be a smart phone, a tablet computer, a personal digital assistant, etc. In some embodiments, the control module may be a processor of the mobile communication device 100 or may be a user guidance information application software loaded on the mobile communication device 100. In some embodiments, the control module may be a sub-control module of an in-vehicle human-computer interaction device. The mobile communication device 100 communicates with the sub-control module wirelessly or by wire via the input module 160. In some embodiments, the control module may be set in a cloud server. Reference Figure 1 , the mobile communication device 100 may include an imaging module 110 , a control module 120 , a display module 130 , and an input module 160 .

[0033] The imaging module 110 may be a camera of a smart phone or tablet computer, which is integrated in the mobile communication device 100 for shooting or capturing images, for example, images of vehicle components. The control module 120 includes a storage unit 122, an identification unit 124, and a processing unit 126. The storage unit 122 stores a vehicle user manual database. The database includes reference images of vehicle components and user guidance information of the components. Vehicle components may include components in a user guide or user manual provided by the vehicle manufacturer. For example, related components may include user-operable components, components that need to be repaired and replaced, etc. The database may be updated. The updated information may include usage information of newly loaded application software and images of components from different perspectives. In some embodiments, the related component database also includes a virtual three-dimensional model of the component. It should be understood that the "user guidance information" mentioned in this specification may refer to any information related to the component, and may generally include introductory information and pictures in a user guide or user manual provided by the vehicle manufacturer, such as the location, operation method, function, maintenance method, precautions, etc. of the component. The present application is not limited thereto, and the user guidance information may also include introductions related to the component edited by the user himself or downloaded from the Internet, etc., which should be considered to be included in the protection scope of the present application.

[0034] The processing unit 126 of the control module 120 may process the position information and the image information. In some embodiments, the processing unit 126 may determine the viewing angle orientation of the imaging module 110 according to the position of the imaging module 110 and the position of the captured component. Further, the processing unit 126 adjusts the three-dimensional stereo model of the relevant component database to be consistent with the viewing angle of the imaging module 110 capturing the component according to the viewing angle orientation, that is, provides a reference image that is substantially consistent with the captured image. The adjusted three-dimensional stereo model or reference image is then transmitted to the recognition unit 124.

[0035] The recognition unit 124 of the control module 120 may compare the image captured by the imaging module 110 with the reference image stored in the storage unit 122, thereby determining the component in the captured image. In some embodiments, the recognition unit 124 may receive the adjusted three-dimensional stereo model from the processing unit 126 as a reference image to more accurately recognize the component in the captured image. The recognition unit 124 may recognize the image using any suitable image recognition technology.

[0036] In some embodiments, processing unit 124 randomly adjusts the three-dimensional stereo model until a reference image is provided that is substantially consistent with the captured image in terms of the visual orientation of imaging module 110 .

[0037] The display module 130 may be a touch screen display integrated on the outer surface of the mobile communication device 100 to display output information and / or images, for example, user guidance information of the component determined by the recognition unit 124. In some embodiments, the display module 130 communicates with the control module 120, and once the recognition unit 124 determines the component in the captured image, the control module 120 sends a signal to the display module 130, so that the display module 130 outputs the captured image and / or its user guidance information.

[0038] In some embodiments, the mobile communication device 100 can communicate with the vehicle-mounted human-machine interaction device 180 through, for example, the control module 120. The user can choose to use the display of the vehicle-mounted human-machine interaction device 180 to display component images and / or user guidance information. For example, when the user inputs an instruction through the input module 160, the control module 120 instructs the human-machine interaction device 180 to output an image and / or user guidance information. For example, the output can be displayed on the display screen of the human-machine interaction device 180. Alternatively, the image and / or user guidance information can be displayed on a head-up display in the car, or projected onto a flat body decorative part or window using projection technology. In this way, it is more convenient for the user to read the user guidance information. It should be understood that in some embodiments, the input module 160 and the display module 130 of the mobile communication device 100 can be integrated, such as a touch display screen of a mobile phone, which can display images or text information, and can also generate a keyboard or link for the user to input instructions.

[0039] In some embodiments, the mobile communication device 100 further includes a voice recognition module 140, such as a microphone of a mobile phone or a microphone of an earphone. During the recognition process, the voice recognition module 140 also transmits the user's voice prompt to the control module 120, such as a keyword of the recognized component. In this way, it is more convenient for the recognition unit 124 of the control unit 120 to accurately and quickly determine the captured component.

[0040] In some embodiments, reference Figure 1 The control module 120 further includes an image detection module 150 that detects the contrast and / or clarity of the image captured by the imaging module 110 .

[0041] When the recognition unit 124 does not determine (e.g., cannot determine, or the similarity value of the recommended component is too low to satisfy the user) the component in the captured image and the image detection module determines that the contrast of the captured image is not within the threshold range, it can be inferred that the ambient brightness during the capture action is too high or too low to cause the obtained image to be unrecognizable, and the control module 120 can instruct the display module 130 to output a prompt to adjust the ambient brightness to capture the image again. Then, the recognition unit 124 recognizes the recaptured image to determine the vehicle component and outputs the corresponding user guidance information.

[0042] Similarly, when the recognition unit 124 does not determine (e.g., cannot determine, or the similarity value of the recommended component is too low to satisfy the user) the component in the captured image and the image detection module determines that the clarity of the captured image is not within the threshold range, it can be inferred that the user's inaccurate focus during the capture action caused the obtained image to be unrecognizable, and the control module 120 can instruct the display module 130 to output a prompt to adjust the focus again to capture the image. Then, the recognition unit 124 recognizes the re-captured image to determine the vehicle component and outputs the corresponding user guidance information.

[0043] In such an embodiment, through the image detection module 150, the mobile communication device 100 can guide the user to adjust the imaging device 110 by outputting prompts when the vehicle component cannot be accurately determined, so as to obtain an updated image for determining the vehicle component. This further improves the recognition function of the mobile communication device 100.

[0044] The mobile communication device 100 that provides user guidance information may further include a communication module 170. The communication module 170 may be a receiver and / or a transmitter that allows the mobile communication device 100 to connect to a network. The communication module 170 may use a wireless communication connection, such as Wi-Fi, cellular tower communication, and / or satellite communication. In some embodiments, the connection to the network can be connected to a control module of a cloud server to perform the function of providing user guidance information. In this way, the user can use the network to identify vehicle components online to obtain user guidance information without having to download an application or software that executes the program of the method of the present application.

[0045] Figure 2A and Figure 2B 1 is a schematic diagram of a mobile communication device 100 providing user guidance information, showing an image captured by an imaging device 110, and an image and information output by a display module 130. A user can start an application software providing user guidance information through an input module 160 of the mobile communication device 100. The input module 160 may be a user interface of the mobile communication device 100, and preferably the input module 160 and the display module 130 are integrated into one component, such as a touch screen display of a mobile phone or a tablet computer. Figure 2A and 2B In some embodiments, the mobile communication device 100 is used to identify a component 210 on a dashboard of a vehicle 200 . Figure 2A The image is shown from the perspective of the imaging module 110. In some embodiments, the captured image may be output on the display module 130. In some embodiments, the display module 130 is used to display information and / or images after recognition, rather than displaying the image when the image is captured.

[0046] refer to Figure 2BAfter the recognition unit 124 determines that the component 210 in the image is a double flash light, the display module 130 displays the image 131 of the double flash light and its user guidance information 133. The image 131 can be an image captured by the user for identification, or a reference image stored in the storage unit 122, which can be freely set by the user according to personal preference. The user guidance information 133 may include multiple items, such as name, location, operation method, function, maintenance tips, etc. These items can be as follows: Figure 2B The information / images shown are displayed in several rows according to the categories, or can be freely set by the user according to personal preferences, such as being displayed in multiple columns, being displayed in a dispersed manner with the image as the center, being displayed randomly, etc. In some embodiments, the display module 130 can be a touch screen of a mobile phone, which is also a user interface for inputting information. The user can process the displayed information / images by operating on the touch screen. For example, the user can store or replace images, edit user guidance information, modify the information display form, etc. The updated images and / or user guidance information can be stored in the storage unit 122 for the convenience of the user's next operation.

[0047] Figure 3A and 3B yes Figure 1 Schematic diagram of a mobile communication device 100, Figure 3A The display module 130 of the mobile communication device 100 displays the information of the component associated with the component 210. Figure 3B The display module 130 of the mobile communication device 100 is shown to display information of three components with high similarity to the component 210. In some embodiments, the three-dimensional stereo model stored in the storage unit 122 provides the position and image information of the components of the vehicle 200. After the recognition unit 124 determines the captured component 210, the control unit 120 can determine the components associated with the component 210 based on the three-dimensional stereo model, so that its user guidance information can also be output and displayed on the display module 130. In some embodiments, the recognition unit 124 provides information of the components associated with the determined position of the component 210 based on the three-dimensional stereo model. Figure 3A , the display module 130 may display components near the location of the component 210, such as arrows in four directions, such as up, down, left, and right, indicating components around the component 210 (e.g., the display screen 212 at the top, the air outlet 214 at the left, the map card slot 216 at the right, and the disk inlet and outlet 218 at the bottom). Related component links may also be provided on the display module 130 for the user to select to access information about the related components.

[0048] The mobile communication device 100 may also provide information about components associated with the identified component functions. In some embodiments, the components associated with the determined component functions may also be output on the display module 130 for user selection (not shown). For example, if the identified component is the air conditioning switch of the vehicle, the display device 130 also outputs images of related buttons such as the wind direction adjustment button, the air volume size button, and the temperature adjustment button, and user guidance information for the user to select and refer to.

[0049] In some embodiments, the recognition unit 124 may select multiple components with high similarity as the search results for display based on the similarity between the reference image of the component in the database and the actual captured image. For example, four components with high similarity are selected, and the display module 130 outputs user guidance information of the multiple components with high similarity. Figure 3B yes Figure 1 FIG. 1 is a schematic diagram of a mobile communication device 100, showing that the display module 130 of the mobile communication device 100 outputs information of three other components that are highly similar to the component 210. In some embodiments, regardless of whether the recognition unit 124 determines the component 210, the display module 130 may output user guidance information of the components that are highly similar to the component 210. Figure 3B , the display module 130 may display information of three other components that are highly similar to the double flash light button, such as the disc player play shortcut key 220, the disc in and out control shortcut key 230, and the door lock button 240. The output information may include component names and links, and images of similar components, etc. In addition, the display module 130 may also display information that the selected component is a possible confirmed component and a similarity value.

[0050] Figure 4A and 4B 3 and 4 respectively show a mobile communication device 300 and a control module 320 thereof for providing user guidance information according to another embodiment of the present application. Figure 4A and 4B, the mobile communication device 300 can be implemented in a portable device such as a smart phone. The mobile communication device 300 includes an imaging module 310 (e.g., a camera of a smart phone), a control module 320, and a display module 330 (e.g., a display screen of a mobile phone). In some embodiments, the control module 320 may be a user guidance information application software loaded on the mobile communication device 300. In some embodiments, the control module 320 may be a sub-control module of the vehicle-mounted human-computer interaction device. The mobile communication device (smart phone) may also include a communication module (not shown) for wireless or wired communication with the sub-control module 320. In some embodiments, the control module 320 may be set in a cloud server. The control module 320 includes a storage unit 322 and an identification unit 324. The storage unit 322 stores reference images of user-related parts of the vehicle 400, and the identification unit 324 compares the image captured by the imaging module 310 with the reference image in the storage unit 322 to determine the parts in the captured image. The storage unit 322 also stores user guidance information of user-related components of the vehicle 400. Once the recognition unit 324 determines the components in the image, the control module 320 can instruct the display module 330 to display the user guidance information corresponding to the determined components, such as the name and function of the components. The recognition unit 324 can use any suitable image recognition technology to recognize the captured image.

[0051] In some embodiments, reference Figure 4B , the storage unit 322 stores at least one set of virtual three-dimensional models 323 of the vehicle as a basis for providing reference images of components. The virtual three-dimensional model 323 includes multiple components of the vehicle, and the direction and size can be adjusted relative to a reference, so that the mobile communication device 300 can visually provide vehicle images of various directions and sizes as reference images. The control module 320 also includes a processing unit 326, which can obtain images of vehicle components in various directions or different perspectives as reference images relative to the mobile communication device 300 by adjusting the direction of the three-dimensional model 323 for comparison with the captured image to determine the components in the captured image. In this way, only one set of three-dimensional models 323 is provided, and reference images of vehicle components in various directions can be obtained, which is more comprehensive than storing a large number of images of different perspectives as reference images, and the situation that the image captured from a special shooting angle cannot be recognized can be avoided and the storage space of the storage unit 322 can be saved.

[0052] Figure 4C and 4DIt is respectively shown that the processing unit 326 can adjust the direction of the three-dimensional model 321 of the vehicle based on the orientation of the mobile communication device 300 or more specifically based on the imaging module 310 relative to the captured component 410 of the vehicle 400, so that it is consistent with the orientation of the component in the captured image, that is, the visually adjusted three-dimensional model 323' can provide a reference image that is substantially the same as the captured image (the size may be the same or different). For example, Figure 4D The image 324 shown in the dashed box is Figure 4A The image captured in the image capture is basically consistent. In the present application, the "orientation" of the first component relative to the second component may represent the angle at which the first component is offset relative to the second component in a certain axis direction of the second component, such as the angle between the line connecting the first component and the second component and the longitudinal axis of the second component. The first component may be a captured component, and the second component may be an imaging module 310. The orientation of the imaging module 310 relative to the captured component 410 may be determined by any suitable technology, which is known to those of ordinary skill in the art, such as by a global positioning system, distance, direction and position sensors, etc. For example, the position of the imaging module 310 may be determined by a positioning device (such as GPS) on the mobile communication device 300, and the position of the component 410 may be determined by a sensing device (such as an optical sensor, an acoustic sensor, or other distance sensor, etc.) on the mobile communication device 300 or the vehicle 400. Based on the positions of the imaging module 310 and the component 410, the processing unit 326 may determine the orientation of the imaging module 310.

[0053] The processing unit 326 can adjust the orientation of the 3D stereo model based on at least three reference axis directions (eg, X-axis, Y-axis, and Z-axis of the vehicle) so that the adjusted 3D stereo model can provide a reference image 324 that is substantially consistent with the captured image.

[0054] In one embodiment, reference Figure 4C Before the recognition action, the three-dimensional model 323 in the storage unit 322 is in an initial state, for example, one side of the vehicle body faces the sight direction, that is, it looks like a left view of the vehicle. Figure 4A , the user uses the imaging module 310 of the mobile communication device 300 to capture an image of the component 410 of the vehicle 400, and the captured image is displayed on the display module 330. In the direction parallel to the X-axis, the imaging module 310 is offset in the -Y direction relative to the component 410 (the direction from the component 410 to the imaging module 310) by a certain angle, that is, the angle between the line connecting the imaging module 310 and the component 410 and the X-axis is α.

[0055] refer to Figure 4DDuring the recognition process, the processing unit 326 obtains the angle information and adjusts the three-dimensional model 323 around the X-axis in the -Y direction by an angle α, even if the angle between the X'-axis in the three-dimensional model 323' and the X-axis before adjustment is α. In one embodiment, the line connecting the imaging module 310 and the component 410 has an angle α with the X-axis, and has other angles with the Y-axis and the Z-axis respectively (not shown). The adjustment of the three-dimensional model 323 in the Y-axis and Z-axis directions can refer to the above-mentioned adjustment relative to the X-axis, so it will not be described again. The processing unit 326 obtains the angle information and adjusts the three-dimensional model 323 in the X-axis, Y-axis and Z-axis directions so that the adjusted three-dimensional model 323' provides a reference image consistent with the viewing angle of the imaging module 310. For example, Figure 4A The image captured by the imaging device 310 is Figure 4D The adjusted three-dimensional model 323' is substantially the same as the reference image 324. In this way, the orientation adjustment of the three-dimensional model 323 on three axes is completed, and it can present substantially the same orientation and visual angle as the components in the captured image, thereby facilitating the recognition unit of the mobile communication device 300 to compare and determine the components to be recognized.

[0056] In some embodiments, the processing unit 326 can adjust the size of the three-dimensional stereo model 323 based on the distance A between the imaging module 310 and the component 410 and the orientation of the imaging module 310 relative to the component 410, so that the corresponding component 410' in the reference image 324 is substantially the same size as the component 410 in the captured image, further facilitating identification. The component 410' in the adjusted three-dimensional stereo model is highly similar to the component 410, so the display module 430 can output the image of either the component 410' or the component 410 together according to the user setting when outputting the user guidance information, that is, output the captured image or the reference image 324.

[0057] Figure 5 The method 500 for outputting user guidance information of a vehicle component according to an embodiment of the present application. At 510, the method 500 receives a captured image of a vehicle component. The action of capturing the image can be completed using a camera provided by a mobile communication device. After the camera captures the image, the image can be stored or not stored and only displayed on the mobile communication device.

[0058] At 520, method 500 includes determining whether the captured image meets the requirements for image recognition. If the answer is yes, method 500 continues to 530. Method 500 may identify vehicle components in the captured image based on stored reference images of user-related components of the vehicle. The above-mentioned mobile communication device may store reference images of multiple components of the vehicle and user guidance information of multiple components. In some embodiments, the storage unit of the mobile communication device may store a virtual three-dimensional model of the vehicle, which provides reference images of multiple components of the vehicle. In some embodiments, the storage unit of the mobile communication device may include a user manual database, which includes introduction information and reference images such as three-dimensional images of vehicle components. The user manual database may also include component images and / or instructions for use of components updated and added by users, vehicle manufacturer servers and / or third-party servers.

[0059] The method 500 may compare the captured image with the reference image through its recognition unit to determine the vehicle component in the captured image.

[0060] In some embodiments, the method 500 may further adjust the orientation of the reference image to facilitate comparison with the captured image. At 530, the method 500 may receive the position information of the captured vehicle component and the position information of the imaging device. At 540, the method 500 determines the orientation of the imaging device relative to the captured vehicle component at the time of capture. The orientation of the imaging device relative to the captured vehicle component may be the angle between the line between the imaging device and the captured vehicle component position and the vehicle coordinate axis.

[0061] At 550, method 500 adjusts the orientation of the vehicle reference image. In some embodiments, the reference image is provided by a virtual three-dimensional model of the vehicle, and the orientation of the three-dimensional model is adjusted in step 550 to provide an adjusted reference image. Method 500 may adjust the orientation of the three-dimensional model according to the orientation of the captured vehicle component so that in the visual direction of the mobile communication device, the adjusted three-dimensional model can provide a reference image consistent with the captured image, which can greatly improve the recognition efficiency of the recognition unit. Alternatively, adjusting the viewing angle orientation of the reference image includes randomly adjusting the orientation of the three-dimensional model until the three-dimensional model provides an image that is substantially consistent with the captured image.

[0062] At 560, the method 500 identifies at least one vehicle component based on the adjusted reference image. In some embodiments, the method 500 identifies a captured vehicle component. In some embodiments, the method 500 identifies several components, such as three or four components, that have a high degree of similarity to the vehicle component in the captured image as the identified vehicle component. Next, at 570, the method 500 outputs user guidance information for at least one identified component. Outputting the user guidance information for at least one identified component includes presenting the user guidance information to the user through an audio device or a display device. In some embodiments, displaying the user guidance information for the identified component includes displaying the name and image of the identified component. In some embodiments, when displaying information for multiple components, links to reference images and / or names of the multiple components are arranged on a display screen of the mobile communication device, and the user can query other user guidance information for each recommended component by clicking on the respective links. The displayed image can be an image captured by the user, or a reference image provided based on a three-dimensional stereo model.

[0063] At 580, method 500 displays information of vehicle components associated with the identified vehicle component, such as images of components that are positionally related (e.g., located near the component) or functionally related (e.g., the components work together or perform similar functions) to the component and / or user guidance information. After 580, method 500 ends or waits for receiving another captured image.

[0064] Returning to step 520, if the method 500 determines that the captured image does not meet the requirements for image recognition, the method 500 proceeds to 590 to instruct the user to recapture the image. At 520, determining that the captured image does not meet the requirements for image recognition includes detecting the contrast and / or clarity of the captured image to determine the quality of the captured image. For example, step 520 includes determining whether the contrast of the captured image is within a threshold range, and if not, indicating at 592 to adjust the ambient brightness and indicating to recapture the image of the vehicle component. Step 520 may also include determining whether the clarity of the captured image is within a threshold range, and if not, indicating at 594 to refocus and indicating to capture the image of the vehicle component again. After 590, the method 500 ends or waits for receiving another captured image.

[0065] In some embodiments, in addition to storing the virtual three-dimensional vehicle model, method 500 also includes storing the captured image as a reference image at any time after the capture action of 510, and after identifying the vehicle parts in the image, the reference image is associated with the identified parts, so that it can be used as an additional reference image to assist recognition in the next recognition action, thereby further improving the recognition accuracy.

[0066] As described above, according to the mobile communication device and the method for outputting user guidance information of vehicle components of the present application, vehicle components can be identified quickly and accurately and user guidance information can be output.

[0067] It should be noted that the method of the present application can be performed by program instructions stored on a computer-readable medium in a mobile communication device. The exemplary control and estimation procedures included herein can be applied together with various vehicle system structures. The specific procedures described here can represent one or more of any number of processing strategies, such as event-driven, interrupt-driven, multi-tasking, multi-threaded processing strategies, etc. Therefore, the various actions, operations or functions shown can be performed in the order shown, in parallel, or can be omitted in some cases. Similarly, the order of processing is not required to achieve the features and advantages of the exemplary embodiments described herein, but is provided for ease of illustration and description. One or more of the actions or functions shown can be performed repeatedly according to the specific strategy used. Moreover, the actions can be graphically represented as codes in a computer-readable storage medium programmed into the vehicle control system.

[0068] It should be understood that the structures and procedures disclosed in the specification of this application are exemplary in themselves, and the specific embodiments should not be understood as limiting, because there may be many variations. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A mobile communication device, comprising: an imaging module configured to capture at least a partial image of a component of the vehicle; A control module, comprising a storage unit, a processing unit and a recognition unit, wherein the storage unit stores reference images of a plurality of vehicle components and user guidance information of the plurality of vehicle components, the processing unit is configured to adjust a viewing angle orientation of the reference image, and the recognition unit is configured to determine a component in a captured image based on the adjusted reference image; and The display module is configured to display user guidance information of the determined component in response to the instruction of the control module. 2 . The mobile communication device according to claim 1 , wherein the reference image in the storage unit is provided by a virtual vehicle three-dimensional stereo model stored in the storage unit.

3. The mobile communication device according to claim 2, wherein the processing unit is configured to: based on the orientation of the imaging module relative to the captured component, adjust the orientation of the three-dimensional stereo model so that it provides a reference image that is substantially consistent with the captured image.

4. The mobile communication device of claim 3, wherein the orientation of the imaging module relative to the captured component is based on: an angle between a line connecting the imaging module and the captured component and an X-axis, a Y-axis, or a Z-axis of the vehicle. 5 . The mobile communication device of claim 2 , wherein the processing unit is configured to randomly adjust the orientation of the three-dimensional stereo model until the three-dimensional stereo model provides a reference image that is substantially consistent with the captured image. 6 . The mobile communication device according to claim 2 , wherein the control module is further configured to instruct the display module to display user guide information of components associated with the determined component. The mobile communication device according to claim 6 , wherein the associated component is a component related to the determined component position.

8. The mobile communication device according to claim 6, wherein the associated component is a component related to the determined component function. 9 . The mobile communication device according to claim 1 , wherein the control module is further configured to instruct the display module to display user guide information of a plurality of components having high similarity to the components in the captured image. 10 . The mobile communication device according to claim 1 , further comprising an image detection module configured to detect contrast and / or clarity of the captured image.

11. The mobile communication device according to claim 10, wherein the control module is further configured to: when the image detection module determines that the contrast of the captured image is not within a threshold range, instruct the display module to output a prompt to adjust the ambient brightness to capture an image again.

12. The mobile communication device according to claim 10, wherein the control module is further configured to: when the image detection module determines that the clarity of the captured image is not within a threshold range, instruct the display module to output a prompt to refocus to capture an image.

13. A method for outputting user guidance information of a vehicle component, comprising: receiving a captured image of the vehicle component; adjusting a viewing angle orientation of a plurality of stored reference images of vehicle components; identifying a vehicle component in the captured image based on the adjusted plurality of reference images of the vehicle component; and User guidance information for the identified vehicle component is output.

14. The method according to claim 13, further comprising: Providing a reference image through a virtual three-dimensional model of the vehicle; determining an orientation of an imaging device used for capturing relative to the vehicle component; Based on the orientation of the imaging device relative to the vehicle component, the orientation of the three-dimensional solid model is adjusted to provide a reference image that is substantially consistent with the captured image.

15. The method of claim 14, wherein the orientation of the imaging device relative to the vehicle component is based on an angle between a line connecting the imaging device and the vehicle component and an X-axis, a Y-axis, or a Z-axis of the vehicle.

16. The method according to claim 13, further comprising: User guidance information of a plurality of vehicle components associated with the recognized component or a plurality of vehicle components having a high degree of similarity to the component of the captured image is output.

17. The method according to claim 13, further comprising: The captured image is detected for contrast and / or sharpness.

18. The method according to claim 17, further comprising: When the contrast of the captured image is not within a threshold range, the user is instructed to adjust the ambient brightness and recapture the image of the vehicle component.

19. The method according to claim 17, further comprising: When the clarity of the captured image is not within a threshold range, the user is instructed to refocus and capture an image of the vehicle component again.

20. The method of claim 13, wherein the output comprises: The image of the identified vehicle component is displayed together with user guidance information related thereto.

Images