Image processing method applied to automobile maintenance and acceptance and related equipment

Abstract

The invention discloses an image processing method applied to automobile maintenance and acceptance and related equipment. The method comprises the steps of obtaining a real automobile image shot by user equipment, loading a target image according to the real automobile image through an AR engine, triggering the user equipment to display according to the real automobile image and the target image and the like. According to the method, AR display can be provided for the user in the automobile maintenance and acceptance process, so that the due visual effect of the maintained automobile can be visually presented in the acceptance process of the user, the user can form a clear and accurate judgment standard and psychological expectation, and the user experience is improved. According to the method, the user can judge the maintenance result of the maintained automobile efficiently and objectively, whether the maintenance of the maintained automobile meets the technical standard specification or not can be checked effectively, potential safety hazards are eliminated in time, the use safety of the automobile is guaranteed, the communication cost between the user and a maintainer can be reduced, and the rights and interests of the user can be guaranteed. The invention is widely applied to the technical field of automobiles.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and in particular to an image processing method and related equipment for automotive maintenance and acceptance. Background Technology

[0002] Cars inevitably experience wear and tear or damage during use, necessitating maintenance at repair shops or factories. Car maintenance includes repairing and replacing damaged or outdated parts, adding consumables such as engine oil and antifreeze, and checking and correcting loose parts. Because car maintenance is highly specialized, the process is usually kept secret from the user. The user only sees the results when the mechanic returns the car, and the mechanic typically only provides a simple conclusion like "repair complete," without specific data to support this claim. Furthermore, without specialized tools like laser rangefinders and torque wrenches, users can only rely on visual inspection to judge whether the maintenance has met requirements. However, since users generally lack professional knowledge, they can only rely on intuitive judgments such as "whether the exterior is smooth" and "whether there are obvious gaps," which can easily lead to misjudgments and hinder effective self-inspection.

[0003] The difficulties faced by users have led to a series of problems. For example, the lack of transparency between maintenance providers and users motivates maintenance providers to perform car maintenance without adhering to technical standards and specifications, resulting in potential quality issues. Users may find it difficult to detect non-standard maintenance results in a timely manner, leading to difficulties in subsequent rights protection and even posing safety hazards. Even when maintenance providers strictly follow technical standards and specifications, users may misunderstand their procedures due to unfamiliarity with their professional operations, increasing communication costs. Summary of the Invention

[0004] To address at least one of the aforementioned technical problems, the present invention aims to provide an image processing method and related equipment applicable to automobile maintenance and acceptance.

[0005] On one hand, embodiments of the present invention include an image processing method for vehicle maintenance and acceptance, the image processing method for vehicle maintenance and acceptance comprising the following steps: Acquire real vehicle images captured by the user's device; the content of the real vehicle images is the entire vehicle or a part of the vehicle being maintained; Run the AR engine; The AR engine loads the corresponding target image based on the real vehicle image. The user device is triggered to display the image based on the actual vehicle image and the target image.

[0006] Furthermore, the step of loading the corresponding target image based on the real vehicle image using the AR engine includes: The AR engine searches the reference model library, which stores multiple standard models corresponding to the car being maintained. When a standard model matching the actual vehicle image is found from the reference model library, the found standard model is determined as the target image.

[0007] Furthermore, the step of loading the corresponding target image based on the real vehicle image using the AR engine includes: Get random numbers; When the random number is the first value, the actual vehicle image itself is used as the target image; When the random number is the second value, the AR engine searches the reference model library; the reference model library stores multiple standard models corresponding to the car being maintained. When a standard model matching the actual vehicle image is found from the reference model library, the found standard model is determined as the target image.

[0008] Furthermore, before searching the reference model library, the step of loading the corresponding target image based on the real vehicle image using the AR engine also includes: Based on the design data of the vehicle being maintained, multiple standard models are established. Each of the aforementioned standard models is stored in the reference model library.

[0009] Furthermore, before searching the reference model library, the step of loading the corresponding target image based on the real vehicle image using the AR engine also includes: The car being maintained in its new condition was photographed to obtain multiple images of the new car; Based on the images of the new vehicles, multiple standard models are established. Each of the aforementioned standard models is stored in the reference model library.

[0010] Further, triggering the user device to display based on the actual vehicle image and the target image includes: Using the AR engine, a first anchor point is established on the target image, and a second anchor point is established on the real vehicle image; the second anchor point corresponds to the position of the first anchor point. Using the AR engine, image tracking of the target image to the real vehicle image is established based on the first anchor point and the second anchor point; The target image is rendered onto the first layer; The actual vehicle image is rendered onto the second layer; The first layer and the second layer are merged to obtain a merged image; The fused image is sent to the user equipment for display.

[0011] Furthermore, the image processing method applied to vehicle maintenance and acceptance also includes: Obtain the repair order information of the vehicle being maintained; Based on the repair order information, determine the parts to be maintained; Based on the maintenance component, detect the first maintenance component region in the actual vehicle image and the second maintenance component region in the target image; Detect the positional deviation between the first maintenance component area and the second maintenance component area; Based on the position deviation value, an acceptance prompt message is generated.

[0012] Furthermore, the image processing method applied to vehicle maintenance and acceptance also includes: The user device is triggered to display a confirmation request message; the confirmation request message is used to request the user to confirm that the current display screen of the user device matches the actual state. The user equipment obtains the user's response operation information to the confirmation request; the response operation information indicates the user's confirmation of the matching status. When the response operation information does not match the current value of the random number, the user device is triggered to display a reminder message.

[0013] On the other hand, embodiments of the present invention also include a computer device, including a memory and a processor, the memory for storing at least one program, and the processor for loading at least one program to execute the image processing method applied to vehicle maintenance acceptance in the embodiments.

[0014] On the other hand, embodiments of the present invention also include a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the image processing method applied to vehicle maintenance acceptance in the embodiments.

[0015] The beneficial effects of this invention are as follows: The image processing method applied to vehicle maintenance acceptance in the embodiments can provide AR display to users during the vehicle maintenance acceptance process. Therefore, it can visually and intuitively present the visual effects that the vehicle being maintained should have during the user's acceptance process, that is, visually and intuitively present the maintenance goals of the vehicle being maintained. This enables users to form clear and accurate judgment criteria and psychological expectations, which is conducive to users efficiently and objectively judging the maintenance results of the vehicle being maintained. It can effectively check whether the maintenance of the vehicle being maintained meets the technical standards and specifications, eliminate safety hazards in a timely manner, ensure the safety of vehicle use, and also reduce the communication costs between users and maintenance providers, thus protecting the rights and interests of users. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of a system that can implement an image processing method for automobile maintenance and acceptance in the embodiment; Figure 2 This is a schematic diagram illustrating the steps of the image processing method applied to vehicle maintenance and acceptance in this embodiment; Figure 3 This is a schematic diagram illustrating the conversion of a new vehicle image into a standard model in an embodiment. Figure 4 This is a schematic diagram illustrating the principle of steps S401-S406 in the embodiment; Figure 5 This is a schematic diagram of the acceptance prompt information in the embodiment; Figure 6 This is a schematic diagram illustrating the principle of steps S301B-S304B in the embodiment. Detailed Implementation

[0017] Terminology Explanation: AR: Augmented Reality, is a technology that combines computer-generated virtual information (such as text, images, etc.) with augmented reality. The technology that seamlessly overlays 3D models, videos, etc. onto the real-world environment observed by the user through computing devices, and enables the two to interact in real time, thereby enhancing the user's perception of the real environment, aims to "empower" the real world and provide an experience that transcends ordinary senses.

[0018] In this embodiment, the image processing method applied to vehicle maintenance and acceptance can be... Figure 1 The system implementation is shown. (Refer to...) Figure 1The system includes a server and user devices. The server can be operated by the car maintenance company, and the user devices can be a mobile phone held by the user or a tablet computer lent to the user by the maintenance company. The user devices are equipped with visible light cameras to capture images using visible light. The user devices may also be equipped with sensors such as LiDAR, gyroscopes, and positioning modules. When the visible light camera is capturing an image, the LiDAR can detect the distance between the user device and the object being photographed, thus obtaining depth information; the gyroscope can detect the pose of the user device, thus obtaining pose information; and the positioning module can locate the user device, thus obtaining its spatial coordinates. The user device can add the depth information, pose information, and spatial coordinates as supplementary information to the visible light image, which can then be used when needed for AR-related processing.

[0019] Maintenance providers can develop apps or mini-programs for users to install and run on their devices, enabling users to communicate with the server via wireless communication protocols such as WiFi or Bluetooth.

[0020] In this embodiment, the APP or mini-program running on the user device includes the following functional modules: Vehicle identification binding module: Automatically matches vehicle model and maintenance items by scanning VIN code or Bluetooth, and accurately retrieves the acceptance standards of corresponding parts; Intelligent shooting guidance module: Real-time display of shooting frame and angle guidance, built-in image quality detection algorithm, automatically prompts problems such as "blurry image" and "angle deviation" to ensure that the shooting meets the comparison requirements; AR Engine: It adopts the AR Foundation cross-platform framework, integrates the spatial positioning capabilities of ARKit (iOS) and ARCore (Android), and achieves accurate registration between virtual models and real scenes through VIO (Visual Inertial Odometry) technology; Results visualization module: Displays the results of the AR processing module in a dual-layer format of "real image + AR annotation". Qualified areas are highlighted in green, while deviations are marked in red, and simplified conclusions are displayed simultaneously.

[0021] When a user takes their car to a maintenance service provider for maintenance, and the service provider notifies the user of the completion and acceptance procedures, they can base their decision on... Figure 1 The system shown performs image processing methods applied to vehicle maintenance acceptance. The vehicle being maintained by the maintenance company is the vehicle being maintained.

[0022] In this embodiment, refer to Figure 2 The image processing method applied to vehicle maintenance and acceptance includes the following steps: S1. Acquire real vehicle images captured by the user's device; S2. Run the AR engine; S3. Using the AR engine, load the corresponding target image based on the real vehicle image; S4. Based on the actual vehicle image and the target image, trigger the user device to display the image.

[0023] In this embodiment, the server can execute the various steps of the image processing method applied to vehicle maintenance and acceptance, including steps S1-S4. During the execution of the image processing method for vehicle maintenance and acceptance, the server can communicate with the user equipment to retrieve data such as images captured by the user equipment, or control the user equipment to execute corresponding steps.

[0024] In this embodiment, a user can handheld the user device to photograph the vehicle being maintained. Specifically, the user can adjust parameters such as the field of view and pose of the user device according to their acceptance requirements to photograph the entire vehicle or parts thereof (e.g., engine, tires, interior, etc.) to obtain a real vehicle image. The displayed content of the real vehicle image is the entire vehicle or its parts. Alternatively, the server can obtain the user's repair order information and determine which parts or components of the vehicle have undergone maintenance based on the repair order information. The user device can then guide the user to photograph the maintained parts.

[0025] In this embodiment, the user equipment captures images of the vehicle being maintained dynamically, meaning multiple frames of real-vehicle images are acquired every second. Since the processing principle for each frame of real-vehicle image is the same, the process for processing one frame of real-vehicle image will be used as an example for explanation. Unless otherwise specified, the real-vehicle image referred to is one frame of real-vehicle image.

[0026] In step S2, the server can run an AR engine based on frameworks such as ARFoundation.

[0027] In this embodiment, when the server executes step S3, which is to load the corresponding target image based on the real vehicle image using the AR engine, the following steps can be performed: S301A. Search the reference model library using the AR engine; S302A. When a standard model matching the actual vehicle image is found in the reference model library, the found standard model is identified as the target image.

[0028] Steps S301A-S302A are the first execution method of step S3.

[0029] In this embodiment, before executing steps S301A-S302A, a step of establishing a reference model library can be performed. Specifically, the manufacturer of the vehicle being maintained can provide the CAD (Computer-Aided Design) design data of the vehicle. Through processes such as Sharp3D modeling and Adobe Aero optimization, the CAD design data is converted into an AR-usable model to obtain a standard model. The standard model describes the spatial position, orientation, and relative relationship with surrounding components of the vehicle being maintained, matching the quality inspection-level accuracy requirements of the vehicle manufacturer. Different standard models can describe different components on the vehicle, or describe the same component from different perspectives and viewing distances. Therefore, labels including component serial numbers, perspectives, and viewing distances can be established, and these labels are used to mark the standard models. These standard models are stored in the reference model library, thereby establishing the reference model library.

[0030] Since CAD design data is the original design data of the car being maintained, it is not affected by errors generated by individuals during production and other processes. Therefore, the standard model generated in this way has high benchmark reference value.

[0031] In this embodiment, at the time when a user newly purchases a car for maintenance, the user can take photos of the entire vehicle or specific components from different angles and viewing distances using their device, obtaining multiple images of the new car. Each new car image corresponds to a specific component, a specific angle, and a specific viewing distance on the car. Therefore, tags including component number, angle, and viewing distance can be created, and these tags can be used to label the new car images. (Refer to...) Figure 3 For each new car image, steps such as feature point extraction and 2D-3D conversion can be performed to convert the new car image into an AR-usable model, obtaining a standard model. The label corresponding to the new car image is used to mark the corresponding standard model. These standard models are stored in a reference model library, thereby establishing the reference model library.

[0032] A standard model is obtained by processing images of the car being maintained in its new condition. Since the standard model is obtained by detecting the car itself and is based on data from the car itself, the standard model generated in this way has good individual matching and comparability with real car images from the same car being maintained.

[0033] After the reference model library is established, steps S301A-S302A can be performed.

[0034] In step S301A, the AR engine running on the server searches the reference model library. Specifically, the AR engine can read additional information such as shooting depth information, shooting pose information, and spatial coordinate information written into the new car image when the user device captures the real car image. Based on this additional information, it searches the reference model library for tags with the same or highest similarity. If such a tag is found, then the standard model corresponding to such a tag is the standard model that matches the real car image, and step S302A can be executed.

[0035] In step S302A, the server identifies the found standard model as the target image, which is then used to execute step S4.

[0036] In this embodiment, when the server executes step S4, which triggers the user device to display the image based on the real vehicle image and the target image, the following steps can be performed: S401. Using the AR engine, a first anchor point is established on the target image, and a second anchor point is established on the actual vehicle image; S402. Using the AR engine, image tracking of the target image to the real vehicle image is established based on the first anchor point and the second anchor point; S403. Render the target image onto the first layer; S404. Render the actual vehicle image onto the second layer; S405. Merge the first and second layers to obtain a merged image; S406. Send the merged image to the user equipment for display.

[0037] The principle of steps S401-S406 is as follows: Figure 4 As shown.

[0038] Reference Figure 4 In step S401, the AR engine establishes multiple first anchor points on the target image and multiple second anchor points on the actual vehicle image. Each first anchor point on the target image corresponds to a second anchor point on the actual vehicle image, and the corresponding first and second anchor points represent the same positions in the actual vehicle image and the target image.

[0039] In step S402, the AR engine establishes a locking relationship between each pair of corresponding first and second anchor points, so that when the real vehicle image undergoes translation, rotation, switching, or other transformations, the target image can also undergo translation, rotation, switching, or other transformations synchronously, thereby achieving image tracking of the target image onto the real vehicle image.

[0040] Reference Figure 4In steps S403-S405, the AR engine renders the target image and the actual vehicle image on the first and second layers respectively, and merges the two layers to obtain a merged image. In step S406, the AR engine sends the merged image to the user device, and the result visualization module in the APP running on the user device displays the merged image, thus showing a two-layer display result of "real image (actual vehicle head) + AR annotation (target image)", realizing AR display in the process of vehicle maintenance and acceptance.

[0041] In this embodiment, by executing steps S1-S4, when using the AR standard model as the target image, AR display can be provided to the user during the vehicle maintenance acceptance process. The standard model can be generated by AR modeling based on the CAD design data of the vehicle being maintained or a new vehicle image in its new state. Therefore, the target image can visually and intuitively present the visual effect that the vehicle being maintained should have during the user's acceptance process, that is, visually and intuitively present the maintenance target of the vehicle being maintained. This enables the user to form clear and accurate judgment standards and psychological expectations, which is conducive to the user's efficient and objective judgment of the maintenance results of the vehicle being maintained. It can effectively check whether the maintenance of the vehicle being maintained meets the technical standards and specifications, eliminate safety hazards in a timely manner, ensure the safety of vehicle use, and also reduce the communication costs between the user and the maintenance provider, thus protecting the user's rights.

[0042] In this embodiment, in addition to performing steps S1-S4, the following steps may also be performed: S5. Obtain the repair order information for the vehicle being maintained; S6. Determine the parts to be maintained based on the maintenance order information; S7. Based on the maintenance component, detect the first maintenance component region in the real vehicle image and the second maintenance component region in the target image; S8. Detect the positional deviation between the first maintenance component area and the second maintenance component area; S9. Generate acceptance prompt information based on the position deviation value.

[0043] In step S5, the server can obtain the repair order information of the car being maintained. The repair order information records which parts of the car the user wants to be maintained, which parts the maintenance provider actually maintains, and other information. These parts that need to be maintained and which parts were actually maintained are the maintenance parts, which can be determined in step S6.

[0044] In step S7, using maintenance components as detection targets, the area in the real vehicle image containing maintenance components (such as the window button on the left rear door) is identified as the first maintenance component area, and the area in the target image containing the same maintenance component (such as the window button on the left rear door) is identified as the second maintenance component area. Specifically, when performing step S7, unique or specific feature points contained in the maintenance components (such as the edge of a bolt nut, the position of a bearing mounting hole, the inflection point of the component's geometric contour, etc.) can be used as detection targets to detect both the real vehicle image and the target image.

[0045] In step S8, the first maintenance component area and the second maintenance component area are regarded as feature points in the real vehicle image and the target image. Through the feature point matching algorithm, information such as the presence or absence of components, translational position offset, and angular offset between the first maintenance component area and the second maintenance component area is calculated to obtain the position deviation value.

[0046] For example, refer to Figure 5 When the repair order information shows that the left rear door's inner door handle and window button were repaired, it can be determined that the repaired components include the inner door handle and window button. From the real vehicle image, the first maintenance component area containing the inner door handle and the second maintenance component area containing the window button are detected. From the target image, the second maintenance component area containing the inner door handle and the third maintenance component area containing the window button are detected. For the first and second maintenance component areas containing the inner door handle, the presence or absence of components (whether both contain the inner door handle), translational offset (the translational distance between the inner door handles in the two areas), and angular offset (the relative rotation angle of the inner door handles in the two areas) are detected. If both components are present, the translational offset is less than the distance tolerance threshold, and the angular offset is less than the angular tolerance threshold, then the position deviation value is considered acceptable. In step S9, if... Figure 5 As shown, a green pattern can be superimposed on the first or second maintenance component area where the inner latch is located in the fused image as part of the acceptance prompt information, indicating that the maintenance result of the inner latch component is qualified; for the first and second maintenance component areas where the window button is located, the presence or absence status of the components in the first and second maintenance component areas (whether both contain the window button component), the translational position offset (the translational distance between the window buttons in the two areas), and the angular offset (the relative rotation angle of the window buttons in the two areas) are detected. If the components are present, but the translational position offset is greater than the distance tolerance threshold or the angular offset is greater than the angular tolerance threshold, then the position deviation value is determined to be too large. In step S9, if... Figure 5As shown, a red pattern can be overlaid on the first or second maintenance component area where the window button is located in the fused image as part of the acceptance prompt information, indicating that the maintenance result of the window button component is unqualified.

[0047] In this embodiment, during step S9, in addition to generating acceptance prompt information, an acceptance report can also be generated based on the acceptance prompt information. The content of the acceptance report may include the acceptance prompt information, the specific value of the position deviation, etc. The user equipment can send the acceptance report to the maintenance provider's repair advisor, thereby triggering the after-sales rework process.

[0048] By executing steps S5-S9, the maintenance results and quality inspection results, down to the component level, can be displayed to the user intuitively, thereby reducing the need for the user to have automotive expertise and assisting the user in the acceptance of vehicle maintenance.

[0049] By executing steps S1-S9, the following user-independent vehicle acceptance process can be achieved: 1. Preparation Phase: Linking Vehicles with Repair Projects After the repair is completed, the customer can open the car manufacturer's after-sales app / mini-program and bind the scenario in two ways: Automatic binding: Repair work orders are synchronized to the APP and directly display "Items to be accepted"; Manual binding: Scan the repair order QR code or enter the VIN code and select the corresponding repair item; The system automatically preloads the AR standard model and acceptance rules required for the project.

[0050] 2. Shooting Stage: Intelligent guidance to acquire qualified images After entering the acceptance interface, the system provides precise guidance based on the characteristics of the parts being repaired: The screen displays a live view frame, overlaid with a dotted frame indicating "recommended shooting angle"; The built-in sensor detects the shooting distance and prompts "Please move closer" and "Keep the phone level"; Automatically detects image clarity and obstruction. If there is oil or insufficient light, it will prompt "Please wipe the lens" or "Turn on the flash". After the customer completes the photo shoot as prompted, the app automatically uploads the image to the cloud-based comparison engine.

[0051] 3. Acceptance Phase: AR Visualization of Results After cloud processing is complete, the app simultaneously displays triple acceptance information: AR overlay view: A virtual standard model is precisely overlaid on the real shooting scene. You can drag and rotate it with two fingers to view details. Deviation areas will automatically flash and the deviation value will be marked. Conclusion Summary: The classification is clearly defined using "Qualified / Needs Adjustment / Unqualified," accompanied by easy-to-understand explanations; Professional data: Clicking "View Details" will display professional parameters (such as torque value, angle deviation, etc.) for customers who need them. If the result is "to be adjusted", the customer can directly send the acceptance report to the maintenance consultant through the APP, which will trigger the after-sales rework process.

[0052] In this embodiment, when performing step S3, which is to load the corresponding target image based on the real vehicle image using the AR engine, the following steps can also be performed: S301B. Get random numbers; S302B. When the random number is the first value, the actual vehicle image itself is used as the target image; S303B. When the random number is the second value, the AR engine searches the reference model library; the reference model library stores multiple standard models corresponding to the car being maintained. S304B. When a standard model matching the actual vehicle image is found in the reference model library, the found standard model is identified as the target image.

[0053] Steps S301B-S304B are the second execution method of step S3. Among them, steps S303B-S304B are the same as the first execution method of step S3, namely steps S301A-S302A.

[0054] Specifically, in step S301B, the shooting parameters of the user device in step S1 can be used as a seed for generating random numbers. For example, if the user changes the shooting field of view of the user device by means of translation, rotation, zoom, etc., causing a change in the content of the real vehicle image obtained in step S1, then the generation of a new random number corresponding to the real vehicle image will be triggered.

[0055] In this embodiment, the random number is a binary random number, which has two possible values: a first value (specifically 0) and a second value (specifically 1). The probabilities of the two values ​​can be the same, that is, each time a random number is generated or switched, there is a 50% probability of obtaining the first value (0) and a 50% probability of obtaining the second value (1). Alternatively, the probability of the first value (specifically 0) can be set to be greater than the probability of the second value (specifically 1). For example, each time a random number is generated or switched, there is a smaller (e.g., 20%) probability of obtaining the first value (0) and a larger (e.g., 80%) probability of obtaining the second value (1).

[0056] In this embodiment, by executing steps S301B-S304B, each time the user device captures a new real vehicle image, step S302B is randomly selected to be executed, using the new real vehicle image itself as the target image. Thus, when executing steps S401-S406, the real vehicle image is fused with itself to obtain a fused image, so that the screen displayed by the user device is the same as the new real vehicle image itself. Alternatively, steps S303B-S304B are randomly selected to be executed, and in accordance with the method of steps S301A-S302A, an AR standard model is found from the reference model library as the target image. Thus, when executing steps S401-S406, the standard model is fused with itself to obtain a fused image, so that the screen displayed by the user device includes the standard model.

[0057] In this embodiment, steps S401-S406 are executed based on steps S301B-S304B, and the resulting effect is as follows: Figure 6 As shown. (Refer to...) Figure 6 As the user device changes its shooting view, it sequentially captures real vehicle images 1, 2, 3, etc., and triggers the synchronous generation of random numbers, obtaining the second value (1), the first value (0), the second value (1), etc.; thus, for the first captured real vehicle image, i.e., real vehicle image 1, the corresponding random number is the second value (1), which will trigger the execution of steps S301B-S304B (steps S301A-S302A), searching for the standard model 1 that matches real vehicle image 1 from the reference model library and fusing it, so that the user device's display screen shows the content containing the standard model of AR; for the first captured real vehicle image, i.e., real vehicle image 1, the corresponding random number is the second value (1), which will trigger the execution of steps S301B-S304B (steps S301A-S302A), searching for the standard model 1 that matches real vehicle image 1 from the reference model library and fusing it, so that the user device's display screen shows the content containing the standard model of AR; for the first real vehicle image 1, the second value (1)... The two captured images of the actual vehicle, namely actual vehicle image 2, have a corresponding random number of the first value (0). This will trigger step S302B, where actual vehicle image 2 itself is used as target image 2 and fused with actual vehicle image 2, so that the content of actual vehicle image 2 itself is displayed on the user device's screen. For the third captured image of the actual vehicle, namely actual vehicle image 3, the corresponding random number is the second value (1). This will trigger steps S301B-S304B (steps S301A-S302A), where a standard model 3 matching actual vehicle image 3 is found in the reference model library and fused with it, so that the content of the standard model containing AR is displayed on the user device's screen. Figure 6 The frame number in the code is used as an example of sequence; for instance, the second frame indicates that it is after the first frame, but it does not necessarily have to be the frame immediately following the first frame. In actual execution, if the user device's field of view does not change significantly, the user device's displayed screen may show the same image for several consecutive frames.

[0058] In this embodiment, based on the execution of steps S301B-S304B, the following steps may also be performed: S10. Trigger the user equipment to display confirmation request information; S11. Obtain user response information to confirmation request information through user equipment; S12. When the response operation information does not match the current value of the random number, the user device is triggered to display a reminder message.

[0059] In step S10, refer to Figure 6 The server can add confirmation request information to every frame of the user's device display, or it can randomly select a portion of the display to add confirmation request information. For example, to... Figure 6 Taking the confirmation request information added to the first frame of the screen as an example, the content of the confirmation request information can be a confirmation window with the text "Does the car image you are seeing now, as captured by your phone, match the actual car you see from outside your phone?" This confirmation window can be set with "match" and "do not match (there is a difference)" confirmation buttons for the user to select and click.

[0060] Guided by the confirmation request, users can observe and judge whether the content currently displayed on their device matches reality (the actual condition of the car as observed directly without using a mobile phone). After making a judgment, users can click the "Match" or "Disagree (Difference Exists)" confirmation button in the confirmation window. The user device generates corresponding response information based on the user's confirmation button click. That is, the response information will be either "Match" or "Disagree (Difference Exists)," indicating the user's confirmation that the current display on the user device matches reality.

[0061] In step S12, the logic shown in Tables 1 and 2 is used to determine whether the response operation information matches the current value of the random number.

[0062] Table 1 shows the logic for obtaining reference values ​​for response operation information.

[0063] First, based on the logic shown in Table 1, a reference value for the response operation information is determined according to the current value of the random number. For example, to... Figure 6 Taking the confirmation request information added to the first frame of the display as an example, the current value of the random number is the value when the first frame of the display is shown, i.e., the second value (1). According to Table 1, the reference value of the response operation information is "inconsistent (there is a difference)". The reference value of the response operation information obtained through the logic of Table 1 is equivalent to the "correct answer" of the response operation information obtained in step S11, and can be used as a benchmark for comparison with the response operation information obtained in step S11.

[0064] Table 2 shows the matching logic between the response operation information and the random number.

[0065] Next, it is determined whether the content of the response operation information obtained in step S11 is the same as the reference value of the response operation information determined by Table 1. Since the reference value of the response operation information has already been determined to be "disagreement (difference exists)," if the content of the response operation information obtained in step S11 is "disagreement (difference exists)," then the content of the response operation information is the same as the reference value. According to Table 2, it is determined that the response operation information matches the random number. In step S12, no processing is required, or the user device can display the message "You're right! Please continue to check your car." If the content of the response operation information obtained in step S11 is "match," then the content of the response operation information is different from the reference value. According to Table 2, it is determined that the response operation information does not match the random number. In step S12, the server can immediately send a reminder message to the user device. The reminder message can specifically be the text "You may have misread it. Please concentrate and continue checking!"

[0066] In this embodiment, the principle of executing steps S10-S12 is as follows: based on executing steps S301B-S304B, it is possible to obtain Figure 6 The display effect shown illustrates that as the user's shooting field of view changes, the value of the random number may change. Therefore, as the value of the random number changes, the display on the user's device will change without notifying the user. Specifically, this occurs... Figure 6 As shown, the display screen can change from including a standard model to including only a real vehicle image, or vice versa. Since the user device's imaging capabilities are sufficient to ensure that the real vehicle image fully reproduces the visual effect of direct human observation of the car, a display screen containing only a real vehicle image objectively presents a visual effect where "the user device's current display screen matches reality." However, the standard model is not the real vehicle image itself, and a display screen including the standard model objectively presents a visual effect where "the user device's current display screen does not match reality (there is a difference)." These objective visual effects are represented as reference values ​​for the response operation information. By requesting the user to return response operation information, the user can be prompted to observe the aforementioned objective visual effects and receive the observation results. When the response operation information does not match the current value of the random number, it indicates that the user has not actually noticed the aforementioned objective visual effects, suggesting that the user may not have carefully inspected the car. By triggering the user device to display a reminder message, the user can be reminded to carefully inspect the car, thereby improving the efficiency and effectiveness of the user's car inspection.

[0067] In this embodiment, the server can also record the cumulative number of times reminder messages are generated. When the cumulative number of generation is greater than or equal to a threshold, the functionality of the app running on the user's device is paused, i.e., steps S1-S4 are suspended. The app needs to be restarted to re-execute these steps. This helps remind users to focus on inspecting the car, improving the efficiency and effectiveness of the user's car inspection.

[0068] The image processing method applied to vehicle maintenance and acceptance in this embodiment has the following effects: 1. For users: Achieve "autonomous, efficient, and accurate" self-inspection. Lower barrier to entry: No professional knowledge required, quick operation, and high self-test success rate; Trust Enhancement: Visualized reports replace "one-way notification from repair shops," increasing the detection rate of installation problems and avoiding "hidden risks." Convenient rights protection: Problems can be reported directly to the repair shop, shortening the rights protection cycle.

[0069] 2. For repair shops: Reduce rework and complaint costs. The rework rate decreased because "users discovered problems in advance," resulting in a lower rework rate for installation-related tasks. Improved reputation: Enhance user trust through "transparent self-inspection".

[0070] 3. For automakers: Strengthen the implementation of after-sales standards Enhanced perception of standards: Users can intuitively experience the "original factory installation standard" through self-inspection, thus increasing brand recognition; Data feedback: By analyzing high-frequency deviations through self-inspection data, we can optimize component design and maintenance training content.

[0071] A computer program can be written to execute the image processing method for vehicle maintenance and acceptance in this embodiment, and then written into a computer device or storage medium. When the computer program is read out and run, the image processing method for vehicle maintenance and acceptance in this embodiment and / or the image processing method for vehicle maintenance and acceptance can be executed, thereby achieving the same technical effect as the image processing method for vehicle maintenance and acceptance in this embodiment and / or the image processing method for vehicle maintenance and acceptance.

[0072] It should be noted that, unless otherwise specified, when a feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. Furthermore, the descriptions of "upper," "lower," "left," and "right" used in this disclosure are only relative to the relative positional relationships of the components of this disclosure in the accompanying drawings. The singular forms "a," "an," and "the" used in this disclosure are also intended to include the plural forms, unless the context clearly indicates otherwise. Moreover, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this embodiment specification is only for describing particular embodiments and is not intended to limit the invention. The term "and / or" as used in this embodiment includes any combination of one or more of the associated listed items.

[0073] It should be understood that although the terms first, second, third, etc., may be used to describe various elements in this disclosure, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, a first element may also be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may also be referred to as a first element. The use of any and all instances or exemplary language (“e.g.,” “such as,” etc.) provided in this embodiment is intended only to better illustrate embodiments of the invention and, unless otherwise required, does not impose a limitation on the scope of the invention.

[0074] It should be recognized that embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can be implemented using standard programming techniques—including a non-transitory computer-readable storage medium configured with a computer program, wherein such a storage medium causes the computer to operate in a specific and predefined manner—according to the methods and drawings described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).

[0075] Furthermore, the procedures described in this embodiment can be performed in any suitable order unless otherwise indicated by this embodiment or otherwise obviously contradict the context. The procedures (or variations and / or combinations thereof) described in this embodiment can be executed under the control of one or more computer systems configured with executable instructions, and can be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. A computer program includes a plurality of instructions executable by one or more processors.

[0076] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices, etc. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. Furthermore, the machine-readable code, or portions thereof, can be transmitted via wired or wireless networks. The invention of this embodiment includes these and other different types of non-transitory computer-readable storage media when such media comprises instructions or programs that implement the steps above in conjunction with a microprocessor or other data processor. When programmed according to the methods and techniques of the invention, the invention also includes the computer itself.

[0077] A computer program can be applied to input data to perform the functions of this embodiment, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects generated on the display.

[0078] The above are merely preferred embodiments of the present invention. The present invention is not limited to the above-described embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention, as long as they achieve the technical effects of the present invention by the same means, should be included within the scope of protection of the present invention. Within the scope of protection of the present invention, the technical solutions and / or implementation methods can have various modifications and variations.

Claims

1. An image processing method applied to vehicle maintenance acceptance, characterized in that, The image processing method applied to vehicle maintenance and acceptance includes: Acquire real vehicle images captured by the user's device; the content of the real vehicle images is the entire vehicle or a part of the vehicle being maintained; Run the AR engine; The AR engine loads the corresponding target image based on the real vehicle image. The user device is triggered to display the image based on the actual vehicle image and the target image.

2. The image processing method for automobile maintenance and acceptance according to claim 1, characterized in that, The step of loading the corresponding target image based on the real vehicle image using the AR engine includes: The AR engine searches the reference model library, which stores multiple standard models corresponding to the car being maintained. When a standard model matching the actual vehicle image is found from the reference model library, the found standard model is determined as the target image.

3. The image processing method for automobile maintenance and acceptance according to claim 1, characterized in that, The step of loading the corresponding target image based on the real vehicle image using the AR engine includes: Get random numbers; When the random number is the first value, the actual vehicle image itself is used as the target image; When the random number is the second value, the AR engine searches the reference model library; the reference model library stores multiple standard models corresponding to the car being maintained. When a standard model matching the actual vehicle image is found from the reference model library, the found standard model is determined as the target image.

4. The image processing method for automobile maintenance and acceptance according to claim 2 or 3, characterized in that, Before searching the reference model library, the step of loading the corresponding target image based on the real vehicle image using the AR engine further includes: Based on the design data of the vehicle being maintained, multiple standard models are established. Each of the aforementioned standard models is stored in the reference model library.

5. The image processing method for automobile maintenance and acceptance according to claim 2 or 3, characterized in that, Before searching the reference model library, the step of loading the corresponding target image based on the real vehicle image using the AR engine further includes: The car being maintained in its new condition was photographed to obtain multiple images of the new car; Based on the images of the new vehicles, multiple standard models are established. Each of the aforementioned standard models is stored in the reference model library.

6. The image processing method for automobile maintenance and acceptance according to claim 2 or 3, characterized in that, The step of triggering the user equipment to display based on the actual vehicle image and the target image includes: Using the AR engine, a first anchor point is established on the target image, and a second anchor point is established on the real vehicle image; the second anchor point corresponds to the position of the first anchor point. Using the AR engine, image tracking of the target image to the real vehicle image is established based on the first anchor point and the second anchor point; The target image is rendered onto the first layer; The actual vehicle image is rendered onto the second layer; The first layer and the second layer are merged to obtain a merged image; The fused image is sent to the user equipment for display.

7. The image processing method for automobile maintenance and acceptance according to claim 6, characterized in that, The image processing method applied to vehicle maintenance and acceptance also includes: Obtain the repair order information of the vehicle being maintained; Based on the repair order information, determine the parts to be maintained; Based on the maintenance component, detect the first maintenance component region in the actual vehicle image and the second maintenance component region in the target image; Detect the positional deviation between the first maintenance component area and the second maintenance component area; Based on the position deviation value, an acceptance prompt message is generated.

8. The image processing method for automobile maintenance and acceptance according to claim 3, characterized in that, The image processing method applied to vehicle maintenance and acceptance also includes: The user device is triggered to display a confirmation request message; the confirmation request message is used to request the user to confirm that the current display screen of the user device matches the actual state. The user equipment obtains the user's response operation information to the confirmation request; the response operation information indicates the user's confirmation of the matching status. When the response operation information does not match the current value of the random number, the user device is triggered to display a reminder message.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory is used to store at least one program and the processor is used to load at least one program to execute the image processing method for vehicle maintenance and acceptance as described in any one of claims 1-8.

10. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the image processing method for automobile maintenance and acceptance as described in any one of claims 1-8.

Images