Vehicle remote diagnosis method and device, terminal equipment and storage medium

Abstract

The invention relates to the technical field of vehicle diagnosis, and relates to a vehicle remote diagnosis method and device, terminal equipment and a storage medium, and the method comprises the steps: when a diagnosis request signal sent by remote equipment is received, analyzing the diagnosis request signal, determining the type of the remote equipment, and according to the diagnosis request signal and the type of the remote equipment, determining the type of the remote equipment; calling a corresponding diagnosis script; acquiring an interface picture of a control interface of the remote equipment in real time; identifying each interaction object icon in the interface picture; according to the diagnosis script, determining a target interaction object icon to be operated, obtaining position data of the target interaction object icon, and fusing the position data and the diagnosis script to obtain an execution script; and sending the execution script to the remote equipment, so that the remote equipment executes the script. Therefore, a merchant can utilize the capability of the identification model to overcome obstacles brought by different types of automobile systems, the business range of diagnosis is improved, and the business range adaptive to remote diagnosis is expanded.

Description

Technical Field

[0001] The present application relates to the field of vehicle diagnosis technology, and in particular to a vehicle remote diagnosis method, apparatus, terminal device and storage medium. Background Art

[0002] In current vehicle remote diagnosis solutions, service providers are often limited to the types of diagnostics they specialize in, leaving them unable to handle those they lack expertise in. For example, some service providers specialize in vehicles like BMWs and Audis, while others specialize in vehicles like Volkswagens and BYDs, leaving them unable to handle other types of vehicles. This significantly impacts the service scope of service providers, limiting their capabilities to their own experience and hindering their ability to expand their business. Summary of the Invention

[0003] In view of this, an embodiment of the present application provides a vehicle remote diagnosis method, which can effectively solve the problem that merchants are unable to expand their business scope.

[0004] In a first aspect, an embodiment of the present application provides a vehicle remote diagnosis method, comprising: When receiving a diagnostic request signal sent by a remote device, parsing the diagnostic request signal, determining the type of the remote device, and calling a corresponding diagnostic script according to the diagnostic request signal and the type of the remote device; Acquire an interface image of the control interface of the remote device in real time; Identify each interactive object icon in the interface image; According to the diagnosis script, determining the target interactive object icon to be operated, obtaining position data of the target interactive object icon, and fusing the position data with the diagnosis script to obtain an execution script; The execution script is sent to the remote device, so that the remote device executes the execution script.

[0005] In some embodiments, when receiving a diagnostic request signal sent by a remote device, parsing the diagnostic request signal to determine the type of the remote device includes: Obtaining diagnostic request content in a data packet of the diagnostic request signal; Parsing the diagnostic request content in the data packet of the diagnostic request signal through a large language model, and determining the corresponding diagnostic service according to the diagnostic request content; the diagnostic request signal is a natural language instruction or a system control instruction; The device identification of the remote device is obtained from the data packet of the diagnosis request signal, and the type of the remote device is determined according to the device identification.

[0006] In some embodiments, the real-time acquisition of an interface image of a control interface of the remote device includes: According to a preset request cycle, a screenshot feedback instruction is sent to the remote device to obtain an interface image of the remote device in real time.

[0007] In some embodiments, determining a target interactive object icon to be operated based on the diagnostic script, obtaining position data of the target interactive object icon, and fusing the position data with the diagnostic script to obtain an execution script includes: Obtaining the interactive object name of the target interactive object icon contained in the diagnostic script; Determining coordinate data of the target interactive object icon in the interface image according to the interactive object name; The coordinate data is written into the diagnosis script to obtain an execution script.

[0008] In some embodiments, after determining the target interactive object icon to be operated, the method further includes: If the target interactive object icon does not exist in the operation interface, the processing of the diagnostic script is suspended and an adjustment instruction is sent to the remote device.

[0009] In some embodiments, identifying each interactive object icon in the interface image includes: The interface image is identified by a pre-trained recognition model, each interactive object icon in the interface image is identified, and the name of each interactive object icon is obtained; the recognition model is trained by the yolov8n model.

[0010] In some embodiments, calling a corresponding diagnostic script according to the diagnostic request signal and the type of the remote device includes: determining an operating system of the remote device according to the type of the remote device; If the operating system of the remote device is an Android system, obtaining an Android adb command corresponding to the diagnostic service, and storing the diagnostic script in a diagnostic array according to the execution order of the diagnostic script; If the operating system of the remote device is a Windows system, the pyautogui command corresponding to the diagnostic service is obtained, and the diagnostic script is stored in a diagnostic array according to the execution order of the diagnostic script.

[0011] In a second aspect, the present application further provides a vehicle remote diagnostic device, comprising: a command receiving module, configured to, upon receiving a diagnostic request signal sent by a remote device, parse the diagnostic request signal, determine the type of the remote device, and call a corresponding diagnostic script based on the diagnostic request signal and the type of the remote device; The interface acquisition module is used to obtain the interface image of the control interface of the remote device in real time; An interface recognition module, configured to recognize each interactive object icon in the interface image; a script integration module, configured to determine, based on the diagnosis script, a target interactive object icon to be operated, obtain position data of the target interactive object icon, and fuse the position data with the diagnosis script to obtain an execution script; The execution module sends the execution script to the remote device so that the remote device executes the execution script.

[0012] In a third aspect, the present application also provides a terminal device, which includes a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the vehicle remote diagnosis method.

[0013] In a fourth aspect, the present application also provides a readable storage medium storing a computer program, which implements the vehicle remote diagnosis method when executed on a processor.

[0014] The embodiments of the present application have the following beneficial effects: This application parses the tester's diagnostic request signal, calls the corresponding diagnostic script, and uses the recognition model to identify the interactive object icons in the current control interface in real time, so that the diagnostic script can correctly act on the interactive object icons on the current control interface, and achieve the technical effect of automatically operating the control interface to complete the test. It allows merchants to use the ability of the recognition model to overcome the obstacles brought by different types of automobile systems, improve the business scope of diagnosis, and increase the business scope of remote diagnosis adaptation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A schematic diagram of a vehicle remote diagnosis method according to an embodiment of the present application is shown; Figure 2 A schematic diagram of a recognition model training process according to an embodiment of the present application is shown; Figure 3 A schematic diagram of a vehicle remote diagnosis scenario according to an embodiment of the present application is shown; Figure 4 A schematic structural diagram of a vehicle remote diagnosis device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0018] The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but rather merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0019] Hereinafter, the terms "including", "having" and their cognates used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the aforementioned items, and should not be understood as excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the aforementioned items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the aforementioned items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and should not be understood as indicating or implying relative importance.

[0020] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.

[0021] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0022] In the current remote diagnosis solution, merchants can only accept remote diagnosis orders for vehicle types they are familiar with on the platform, and cannot handle vehicle types they are not familiar with. This application provides a vehicle remote diagnosis method, which obtains a screenshot of the interface of the remote device, combines it with the recognition model to identify the interface, and then calls the script to remotely control the remote device to perform testing and diagnosis. Through the recognition ability of the recognition model, it can handle areas that the merchant is not familiar with, thereby increasing the business scope of the merchant.

[0023] The vehicle remote diagnosis method is described below with reference to some specific embodiments.

[0024] Figure 1 A flow chart of a vehicle remote diagnosis method according to an embodiment of the present application is shown. Exemplarily, the vehicle remote diagnosis method includes: Step S100 , when a diagnosis request signal sent by a remote device is received, the diagnosis request signal and the type of the remote device are parsed, and a corresponding diagnosis script is called according to the diagnosis request signal and the type of the remote device.

[0025] The vehicle remote diagnostic method of this embodiment is used in scenarios where a merchant's server and a remote customer need to perform vehicle diagnostics. The server uses a remote device on the vehicle to perform diagnostics on the connected vehicle. The remote device can be an onboard computer or onboard diagnostic equipment. During remote diagnostics, the merchant's server controls the remote device to collect vehicle data for diagnostic analysis.

[0026] For example, when a car user needs to diagnose their vehicle, they need to use specialized automotive diagnostic equipment connected to the vehicle for diagnosis. However, if the car user does not know how to diagnose, they can rely on the networking function of the automotive diagnostic equipment to perform remote diagnosis, contacting a vendor that provides remote diagnosis services to assist them in performing vehicle diagnostic operations. The automotive diagnostic equipment can be an onboard computer within the car, or it can be a standalone diagnostic device that is not normally connected to the vehicle and is only connected to the vehicle when diagnosis is required. This automotive diagnostic equipment is the remote device in this embodiment.

[0027] When the server receives the diagnosis request signal sent by the remote device, it will parse the diagnosis request signal and parse the type of the remote device.

[0028] The diagnostic request signal is sent to the server in the form of a datagram. This datagram carries not only the diagnostic request content initiated by the vehicle user but also device information from the remote device. This device information is defined by the communication protocol. By parsing this information in the datagram, the remote device's type can be determined, such as the type of vehicle and operating system. The different contents of the datagram are bound together and stored in specific data bits using different fields. This data can then be successfully parsed according to the established communication protocol.

[0029] The diagnostic request content may be a regular operation instruction sent by the customer through the system of the remote device, such as a comprehensive inspection operation instruction sent through a comprehensive inspection button in the operation interface.

[0030] Furthermore, diagnostic requests can also be in natural language. For example, a user can issue a diagnostic request through voice or text input, such as "Hello, I want to perform a system scan on my 2023 Audi A6 to check for faults." This type of natural language can be processed by a pre-trained large model to extract keywords. For example, "Audi, A6L, 2023, system scan" indicates that the system scan for the 2023 Audi A6L should be selected for automated diagnosis. Consequently, the diagnostic script for the system scan task for the 2023 Audi A6L will be invoked.

[0031] The large language model of this embodiment can be obtained by training based on existing large language models, such as GLM-4-9B-Chat, deepseek and other large language models.

[0032] For example, to currently support the detection of vehicles such as Audi, it is only necessary to establish training data related to this type of vehicle. For example, the previous example "I want to perform a system scan on the 2023 Audi A6 to see if there are any fault codes" can be used as training data, and the recognized keywords "Audi, A6L, 2023, system scan" can be used as the ideal output of the natural language. In this way, multiple training data can be established to input into the large language model for training, so that the large language model can specifically obtain vehicle information and test information in natural language, can smoothly understand the tester's meaning, and call subsequent diagnostic scripts based on the understanding results.

[0033] The large language model can also communicate with users on remote devices to achieve corresponding communication operations.

[0034] It is understandable that different car models have different systems, so the test cases, methods or scripts for automated diagnosis are different. By identifying the diagnostic scripts mentioned above, it is possible to determine what type of car model needs to be processed, so that the correct system scanning service can be selected.

[0035] The diagnostic scripts used in this embodiment to perform these test services are pre-written diagnostic scripts. These diagnostic scripts are written based on the corresponding vehicle model and test service. For example, a series of test scripts are written using the Android adb command. These test scripts are associated with the vehicle model and the corresponding test service, so these diagnostic scripts can be directly retrieved and called.

[0036] For example, if the vehicle diagnostic device's operating system is Android, the diagnostic script can be written using Android's adb command. If the vehicle diagnostic device's operating system is Linux, the diagnostic script can be written in shall. If it is Windows, the pyautogui command in Python can be used to write the command.

[0037] After these diagnostic scripts are obtained, they can be cached and wait for subsequent calls, such as being stored in a diagnostic array. When they are needed later, these scripts can be called directly from the array.

[0038] In addition to differences in operating systems, remote devices are either vendor-provided diagnostic devices or third-party standalone devices. Different devices use different operating systems. If the device is a vendor-provided diagnostic device, you can directly use the corresponding commands. For example, if the vendor-provided devices are all running Android, you can directly use diagnostic scripts written using Android's adb command. For other third-party devices, you need to further determine the operating system used and then determine the appropriate language for the commands.

[0039] Step S200: Acquire an interface image of a control interface of a remote device in real time.

[0040] The various diagnostic scripts in this embodiment mainly simulate various manual click, slide and input operations on the interface, so it is necessary to identify the current remote control interface to determine what should be done currently.

[0041] For example, the method of obtaining the interface image can be to send a screenshot feedback instruction from the server to the remote device to obtain the interface image of the remote device's control interface in real time, or to send a screen recording instruction to obtain a real-time interface video stream, and then obtain key frames from the video stream to obtain the interface image.

[0042] The screenshot feedback command is sent to the remote device at a preset period, such as once every 1 millisecond. If it is a screen recording command, it can be sent after receiving the diagnostic request signal from the remote device, and then continue to receive the video stream.

[0043] In addition, you can also determine whether to take a screenshot or record the screen by identifying whether an operation has been performed on the control interface. If no operation has been performed on the control interface for a long time, the control interface will not change or jump, and there is no need to continue taking screenshots or recording screens. When an operation occurs on the control interface, you can start recording or taking screenshots for a preset time, such as a 2-minute screenshot operation. Each time you operate the operation interface, the preset time will be refreshed.

[0044] The above-mentioned screenshot feedback instruction includes a screenshot instruction and a feedback instruction for feeding back the interface image to the server after the screenshot is taken, so as to control the remote device to periodically take screenshots and feed back images.

[0045] Step S300: Identify each interactive object icon in the interface image.

[0046] After acquiring the image of the control interface, it is necessary to identify the interactive object icons therein. In this embodiment, the interactive object icons in the control interface are identified through an image recognition model.

[0047] Interactive object icons are primarily components that users can interact with using the mouse and keyboard, such as buttons, input boxes, and drop-down menus. After recognition, the recognition model outputs the name and corresponding coordinate data of each interactive object in the interface image.

[0048] Among them, the recognition model is trained by the yolov8n model, and its training process is as follows Figure 2 Shown, including: Step S310 : acquiring an operation interface image of a diagnostic system, and annotating each interactive object icon on the operation interface image to obtain an annotation file.

[0049] Because the test is mainly carried out by the test system in the remote device, and the operation interface is basically on the remote device, it is necessary to collect the operation interface images of each diagnostic system tested. There will be multiple interactive object icons in the image, which need to be labeled.

[0050] For example, if the interactive object icon is a button, the label can be similar to: page_1_button_1, page_2_button_1, page_3_button_1, page_4_button_1, page_4_button_2. The label name indicates the page where the interactive object icon is located and the corresponding label number, etc. Other labeling methods can also be used.

[0051] In this way, each interactive object icon in each interface can be uniquely labeled.

[0052] The above example only uses buttons as an example. Other types of interactive object icons also need to be labeled and used as training data for training.

[0053] Step S320: input the annotation file and the operation interface image into the yolov8n model as training data, perform training for a preset number of iterations, and obtain a recognition model.

[0054] The annotation file and the operation interface image are used as training data, input into the yolov8n model, and the number of iterations is set to perform training. In addition, the recognition model can also be obtained by training other image recognition models.

[0055] During the training process, the output and labeled images can be used to calculate the loss function. When the loss value calculated by the loss function is less than the preset value or the number of iterations is reached, the training is considered successful.

[0056] The resulting recognition model, after inputting a control interface image, can output the labels of each interactive object icon in the interface, thereby determining the type and location of each interactive object icon. Therefore, the recognition model of this embodiment can identify the labels of each interactive object icon on the interface when recognizing the operation interface image. By recognizing the labels of each component, the coordinate position of each interactive object icon on the control interface can be determined.

[0057] It is understandable that through the above-mentioned training method, it is possible to add recognizable test systems to the recognition model. For example, if there are 10 test systems in the current market, then as long as all the operation interface images of these 10 test systems are obtained and trained through the above-mentioned annotation and training methods, the recognition model can successfully recognize the interfaces of these 10 test systems. Therefore, when new test systems, new vehicle models, etc. that provide new test system interface elements appear on the market, the recognition model can be expanded by adding test data. In this way, the diagnostic personnel do not need to be completely familiar with the test system or the vehicle itself. They only need to train the recognition model and write corresponding test cases. Therefore, the method of this embodiment can also better adapt to changes in the market environment and can keep pace with the times.

[0058] Step S400: Determine the target interactive object icon to be operated according to the diagnosis script, obtain the position data of the target interactive object icon, and fuse the position data with the diagnosis script to obtain an execution script.

[0059] In addition to the names of the identified interactive object icons, their positions on the control interface can also be obtained. For example, the center coordinates of each interactive object icon can be directly obtained to locate each interactive object icon. The coordinates can be based on an image coordinate system established with the upper left corner of the current interface image as the origin.

[0060] The coordinates can be directly obtained by the recognition model. When the recognition model recognizes the target interactive object icon in the image, the corresponding center coordinates can be directly obtained according to the current resolution of the image.

[0061] Once the center coordinates are obtained, they can be passed to the diagnostic script obtained in the previous steps.

[0062] The diagnostic script will contain the names of the target interactive object icons that need to be operated. These names are the same as the labels recognized by the recognition model for each interactive object icon. Therefore, when executing the diagnostic script, the names of the interactive objects involved in the diagnostic script will be obtained first, and then these names will be compared with the interactive object icons recognized by the recognition model. If it is determined that all the interactive object icons in the diagnostic script appear on the interface, it means that the currently executed steps are correct, and then the recognition model will feedback the coordinates of these interactive object icons to the program that executes the diagnostic script.

[0063] For example, a diagnostic script might have a statement like "adb input tap page_2_button_2," which means you want to click the button labeled page_2_button_2. The recognition model can then identify the button in the current control interface and provide the coordinates of the button. These coordinates will be passed to the diagnostic script, allowing the click event for the button labeled "page_2_button_2" to be triggered.

[0064] "page_2_button_2" can be a replaceable label. When writing coordinates, the coordinates can replace the name. The specific method for writing coordinates into the script depends on the script statement rules and is not specified here.

[0065] It's important to note that the coordinates of these buttons aren't originally included in the diagnostic script. Image recognition determines the coordinates, which are then fed back to the diagnostic script, allowing it to correctly operate the buttons on the interface. Therefore, the acquired coordinates and the corresponding diagnostic script are integrated to form a valid execution script.

[0066] Among them, there is also a situation where the target interactive object icon involved in the current execution script does not exist in the current interface. At this time, there are the following possibilities: the first possibility is that the current interface is wrong; the second possibility is that the steps of the current execution script are wrong.

[0067] It is known that, in either case, the currently scheduled execution script cannot be executed correctly, so a prompt can be sent to the remote device, requiring the user to switch interfaces, and the current execution script is suspended.

[0068] You can also call a new execution script after determining the current interface, automatically control the interface of the remote device to the correct interface, and then call the previous execution script for integration to ensure the correct execution of the execution script.

[0069] It should be noted that after the user sends a request, the current interface of the remote device should be fixed. However, due to the response delay between the server and the remote device, the user may perform other operations during this period, causing the interface to change, making the current interface not an ideal interface, resulting in the above-mentioned situation where the target interactive object icon in the interface image cannot be obtained.

[0070] Step S500: Send the execution script to the remote device, so that the remote device executes the execution script.

[0071] The diagnostic script needs to be executed on the remote device, so the diagnostic script integrated with the coordinates is sent to the remote device, and the remote device can directly execute it after receiving the diagnostic request signal.

[0072] It should be noted that the execution script of this embodiment is an operation script that specifically performs anthropomorphic operations on the operation interface, such as first clicking the first button of the first interface, then clicking the second button of the second interface, and finally entering the specified command in the first input box of the third interface to start the test. In this way, the remote automated diagnostic operation is completed. It can be seen that this method can be adapted to a variety of vehicles and a variety of diagnostic systems. As long as the interface of the diagnostic system has been learned by the recognition model and the corresponding script has been written, the corresponding remote diagnostic operation can be performed. There is no need for the staff to be familiar with these systems, nor is there a need for the staff to be familiar with the vehicles. The solution of this embodiment also has good scalability. When new diagnostic systems and new vehicles appear, as long as their officially disclosed training processes are obtained and the recognition model is allowed to learn the various interface pictures of their systems, it is possible to adapt to the newly appeared vehicles for automatic diagnosis operations.

[0073] For better explanation of the exemplary Figure 3 , which is a schematic diagram of an operating scenario involving the vehicle remote diagnosis method of this embodiment.

[0074] The user of the remote device sends a diagnostic request signal to the server, and the server performs the above-mentioned remote diagnostic steps. First, the diagnostic request signal is parsed by the large language model to determine what kind of test to perform, and then the corresponding diagnostic script is called to perform remote automated diagnostic operations.

[0075] After calling the diagnostic script, the server will obtain the interface image of the remote device's operating interface in real time. The recognition model will recognize the currently obtained interface image in real time and obtain the interactive object names in the diagnostic script. These names will be matched with the interactive object names recognized in the current interface. If all match, the diagnostic script can be executed. At the same time, the positions of all interactive object icons involved in the diagnostic script will be fed back. In this way, the diagnostic script can correctly operate on the interactive object icons involved, completing the assembly of the diagnostic script. Finally, the assembled diagnostic script is sent to the remote device, which automatically executes these commands, thus realizing remote automated diagnosis of vehicle operations.

[0076] The vehicle remote diagnostic method of this embodiment uses a recognition model and a large language model to identify the remote user's testing requirements and automatically recognize the remote device's operating interface, thereby invoking the correct diagnostic script and remotely controlling the remote device to perform vehicle diagnostic operations. Furthermore, this remote diagnostic method reduces manual diagnostic work and lowers the diagnostic threshold. As long as the recognition model's training data contains the remote device's corresponding operating interface, the corresponding recognition operation can be performed, thereby expanding the service provider's business scope.

[0077] Figure 4An automobile remote diagnosis device according to an embodiment of the present application is shown, comprising: The command receiving module 10 is configured to, upon receiving a diagnostic request signal sent by a remote device, parse the diagnostic request signal and the type of the remote device, and call a corresponding diagnostic script according to the diagnostic request signal and the type of the remote device; The interface acquisition module 20 is used to acquire the interface image of the control interface of the remote device in real time; An interface recognition module 30 is used to recognize each interactive object icon in the interface image; The script integration module 40 is used to determine the target interactive object icon to be operated according to the diagnosis script, obtain the position data of the target interactive object icon, and fuse the position data with the diagnosis script to obtain an execution script; The execution module 50 sends the execution script to the remote device, so that the remote device executes the execution script.

[0078] It can be understood that the device of this embodiment corresponds to the vehicle remote diagnosis method of the above embodiment, and the options in the above embodiment are also applicable to this embodiment, so they will not be described again here.

[0079] The present application also provides a terminal device. Exemplarily, the terminal device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the terminal device to execute the functions of the various modules in the above-mentioned vehicle remote diagnosis method or the above-mentioned vehicle remote diagnosis device.

[0080] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, etc., and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0081] The memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), etc. The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving an execution instruction.

[0082] The present application also provides a readable storage medium for storing the computer program used in the above-mentioned terminal device.

[0083] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. It should also be noted that each box in the structure diagram and / or the flowchart, and the combination of the boxes in the structure diagram and / or the flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0084] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a smart phone, personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0085] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A vehicle remote diagnosis method, characterized in that: include: When receiving a diagnostic request signal sent by a remote device, parsing the diagnostic request signal, determining the type of the remote device, and calling a corresponding diagnostic script according to the diagnostic request signal and the type of the remote device; Acquire an interface image of the control interface of the remote device in real time; Identify each interactive object icon in the interface image; Determine the target interactive object icon to be operated according to the diagnosis script, obtain position data of the target interactive object icon, and fuse the position data with the diagnosis script to obtain an execution script; The execution script is sent to the remote device, so that the remote device executes the execution script.

2. The vehicle remote diagnosis method according to claim 1, characterized in that: When receiving a diagnostic request signal sent by a remote device, parsing the diagnostic request signal to determine the type of the remote device includes: Obtaining diagnostic request content in a data packet of the diagnostic request signal; Parsing the diagnostic request content in the data packet of the diagnostic request signal through a large language model, and determining the corresponding diagnostic service according to the diagnostic request content; the diagnostic request signal is a natural language instruction or a system control instruction; The device identification of the remote device is obtained from the data packet of the diagnosis request signal, and the type of the remote device is determined according to the device identification.

3. The vehicle remote diagnosis method according to claim 1, characterized in that: The real-time acquisition of an interface image of a control interface of the remote device includes: According to a preset request cycle, a screenshot feedback instruction is sent to the remote device to obtain an interface image of the remote device in real time.

4. The vehicle remote diagnosis method according to claim 1, characterized in that: The step of determining a target interactive object icon to be operated according to the diagnosis script, obtaining position data of the target interactive object icon, and fusing the position data with the diagnosis script to obtain an execution script includes: Obtaining the interactive object name of the target interactive object icon contained in the diagnostic script; Determining coordinate data of the target interactive object icon in the interface image according to the interactive object name; The coordinate data is written into the diagnosis script to obtain an execution script.

5. The vehicle remote diagnosis method according to claim 1, characterized in that: After determining the target interactive object icon to be operated, the method further includes: If the target interactive object icon does not exist in the operation interface, the processing of the diagnostic script is suspended and an adjustment instruction is sent to the remote device.

6. The vehicle remote diagnosis method according to claim 1, characterized in that: The identifying each interactive object icon in the interface picture includes: The interface image is identified by a pre-trained recognition model, each interactive object icon in the interface image is identified, and the name of each interactive object icon is obtained; the recognition model is trained by the yolov8n model.

7. The vehicle remote diagnosis method according to claim 3, characterized in that: The calling of a corresponding diagnostic script according to the diagnostic request signal and the type of the remote device includes: determining an operating system of the remote device according to the type of the remote device; If the operating system of the remote device is an Android system, obtaining an Android adb command corresponding to the diagnostic service, and storing the diagnostic script in a diagnostic array according to the execution order of the diagnostic script; If the operating system of the remote device is a Windows system, the pyautogui command corresponding to the diagnostic service is obtained, and the diagnostic script is stored in a diagnostic array according to the execution order of the diagnostic script.

8. A vehicle remote diagnostic device, characterized in that: include: a command receiving module, configured to, upon receiving a diagnostic request signal sent by a remote device, parse the diagnostic request signal, determine the type of the remote device, and call a corresponding diagnostic script based on the diagnostic request signal and the type of the remote device; The interface acquisition module is used to obtain the interface image of the control interface of the remote device in real time; An interface recognition module, configured to recognize each interactive object icon in the interface image; a script integration module, configured to determine, based on the diagnosis script, a target interactive object icon to be operated, obtain position data of the target interactive object icon, and fuse the position data with the diagnosis script to obtain an execution script; The execution module sends the execution script to the remote device so that the remote device executes the execution script.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the vehicle remote diagnosis method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The device stores a computer program, which, when executed on a processor, implements the vehicle remote diagnosis method according to any one of claims 1 to 7.

Images