Method, device and equipment for carrying out quality inspection on vehicle interiors

Abstract

The invention provides a method, a device and equipment for carrying out quality inspection on vehicle interiors. The method is applied to an edge device in the vehicle interior quality inspection system, the vehicle interior quality inspection system further comprises an end side device and a cloud platform which are connected with the edge device through a wireless network, the end side device collects picture data of the vehicle interior, and the cloud platform conducts quality inspection on the vehicle interior. The method comprises the following steps: receiving picture data of the vehicle interior from end side equipment; determining a current transmission strategy of the picture data from a plurality of transmission strategies according to the wireless network quality, the plurality of transmission strategies corresponding to a plurality of data transmission quantities; and transmitting at least part of data in the picture data to the cloud platform according to the current transmission strategy, so that the cloud platform performs quality inspection on the vehicle interior based on at least part of the data. The transmission strategy of the vehicle interior detection picture is determined according to the wireless network quality, and the data volume transmitted to the cloud platform is determined according to the transmission strategy, so that the loss of quality inspection precision can be reduced under the condition that the network quality is poor.

Description

Technical Field

[0001] The present application relates to the field of vehicles, and more specifically, to a method, device and equipment for quality inspection of vehicle interior. Background Art

[0002] Before a vehicle leaves the factory, in order to ensure the functionality, safety, comfort, etc. of the vehicle, it is necessary to conduct quality inspection on the vehicle interior. Vehicle quality inspection involves real-time inspection of more than 100 parts, so it has very high requirements for photo resolution, network transmission, and computing power of image recognition examples. The current basic practice in the industry is to transmit data through industrial wired network connections between end-side devices and edge devices, and edge devices and cloud platforms. This solution has problems such as inflexible deployment, poor scalability, complex software updates and management, data islands, and fixed network resources. In order to overcome these problems, relevant technologies propose to solve them through industrial 5G wireless communication networks. Industrial 5G wireless networks have the characteristics of high bandwidth, low latency, and large-scale connections, which can provide reliable and efficient communication solutions for vehicle interior quality inspection. However, when the quality of the wireless network is poor, image data will be lost during transmission, which will lead to a decrease in the overall quality inspection accuracy of the vehicle interior. Therefore, how to reduce the loss of quality inspection accuracy when the quality of the wireless network is poor is an urgent problem to be solved. Summary of the invention

[0003] The present application provides a method for quality inspection of vehicle interior, which can reduce the loss of quality inspection accuracy when the network quality is poor.

[0004] In a first aspect, a method for quality inspection of vehicle interior is provided, the method being applied to an edge device in a vehicle interior quality inspection system, the vehicle interior quality inspection system also comprising an end-side device and a cloud platform connected to the edge device via a wireless network, the end-side device being used to collect image data of the vehicle interior, the cloud platform being used to quality inspect the vehicle interior, the method comprising: receiving image data of the vehicle interior from the end-side device; determining a current transmission strategy for the image data from a plurality of transmission strategies based on wireless network quality, the plurality of transmission strategies corresponding to a plurality of data transmission volumes, respectively; transmitting at least part of the image data to the cloud platform based on the current transmission strategy, so that the cloud platform performs quality inspection on the vehicle interior based on the at least part of the data.

[0005] In one possible implementation, determining the current transmission strategy of the image data from a plurality of transmission strategies based on the wireless network quality includes: in response to the wireless network quality satisfying a first condition, using a first transmission strategy among the plurality of transmission strategies; in response to the wireless network quality satisfying a second condition, using a second transmission strategy among the plurality of transmission strategies; wherein the wireless network quality corresponding to the first condition is higher than the wireless network quality corresponding to the second condition; the number of images transmitted by the first transmission strategy is greater than the number of images transmitted by the second transmission strategy; and / or the resolution of the image data transmitted by the first transmission strategy is greater than the resolution of the image data transmitted by the second transmission strategy.

[0006] In a possible implementation, the terminal device transmits all the data in the image data to the cloud platform.

[0007] In one possible implementation, transmitting at least part of the image data to the cloud platform according to the current transmission strategy includes: preprocessing at least part of the data to determine at least part of the preprocessed data; and transmitting at least part of the preprocessed data to the cloud platform.

[0008] In one possible implementation, the at least part of the data is a first part of the data in the image data, the image data also includes a second part of the data, and the method further includes: performing quality inspection on the vehicle interior at the edge device based on the second part of the data.

[0009] In a possible implementation, the method further includes: in response to the wireless network quality satisfying a third condition, determining that the data volume of the first part of the data is a first data volume; in response to the wireless network quality satisfying a fourth condition, determining that the data volume of the first part of the data is a second data volume; wherein the wireless network quality corresponding to the third condition is higher than the wireless network quality corresponding to the fourth condition, and the first data volume is greater than the second data volume.

[0010] In a possible implementation manner, the method further includes: subscribing to the prediction service of the wireless network quality from a service module; and receiving the prediction information of the wireless network quality pushed by the service module.

[0011] In a second aspect, a device for quality inspection of vehicle interior is provided, wherein the device is deployed in an edge device in a vehicle interior quality inspection system, wherein the vehicle interior quality inspection system further comprises an end-side device and a cloud platform connected to the edge device via a wireless network, wherein the end-side device is used to collect image data of the vehicle interior, and the cloud platform is used to quality inspect the vehicle interior, wherein the device comprises: a first communication module, used to receive image data of the vehicle interior from the end-side device; a determination module, used to determine a current transmission strategy of the image data from a plurality of transmission strategies according to wireless network quality, wherein the plurality of transmission strategies respectively correspond to a plurality of data transmission volumes; and a second communication module, used to transmit at least part of the image data to the cloud platform according to the current transmission strategy, so that the cloud platform can quality inspect the vehicle interior based on the at least part of the data.

[0012] In a third aspect, an edge device is provided, comprising: a memory for storing a program; and a processor for executing the program stored in the memory, so that the edge device executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0013] According to a fourth aspect, a computer-readable storage medium is provided, on which a program for executing the method described in the first aspect or any possible implementation of the first aspect is stored.

[0014] In a fifth aspect, a vehicle interior quality inspection system is provided, comprising: a terminal device, an edge device and a cloud platform, wherein the edge device is connected to the terminal device and the cloud platform via a wireless network, and the edge device is used to execute the first aspect or any possible implementation method in the first aspect.

[0015] In the embodiment of the present application, a transmission strategy for image data is determined according to the quality of the wireless network, and different strategies correspond to different data transmission volumes. Therefore, the loss of vehicle interior quality inspection accuracy caused by poor network quality can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0017] Figure 1 This is an example diagram of the traditional architecture of a vehicle interior quality inspection system.

[0018] Figure 2 This is an example diagram of a vehicle interior quality inspection solution based on a 5G wireless network.

[0019] Figure 3 It is a schematic flow chart of a method for quality inspection of vehicle interior provided in an embodiment of the present application.

[0020] Figure 4 This is an example diagram of an original image captured by a terminal device provided in an embodiment of the present application.

[0021] Figure 5 It is a schematic flowchart of a method for predicting wireless network quality provided by an embodiment of the present application.

[0022] Figure 6 This is an example diagram of the network quality prediction process provided in an embodiment of the present application.

[0023] Figure 7 This is an example diagram of the vehicle interior quality inspection solution provided in an embodiment of the present application.

[0024] Figure 8 This is an example diagram of another solution for vehicle interior quality inspection provided in an embodiment of the present application.

[0025] Fig. 9 It is a structural diagram of a device for quality inspection of vehicle interior provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0027] Before a vehicle leaves the factory, it is necessary to conduct quality inspection on the vehicle interior to ensure its functionality, safety, comfort, etc. The vehicle interior refers to all the decorative and functional components inside the vehicle. For example, it may include the interior carpet, the front windshield defrost cover, speakers, etc. The vehicle interior quality inspection involves the real-time inspection of more than 100 parts, so it has very high requirements on the photo resolution, network transmission, and computing power of the image recognition algorithm.

[0028] At present, the industry's vehicle interior quality inspection system can include: end-side devices, edge devices and cloud platforms. End-side devices refer to devices that come into contact with the object being inspected and collect data. End-side devices can be cameras, for example. Edge devices refer to devices deployed at the edge nodes of the network. Edge devices are close to the source of data and can perform a certain degree of calculation, data processing and storage. Edge devices can be, for example, algorithm engine platforms. Cloud platforms refer to cloud computing platforms. Cloud platforms are usually concentrated in remote data centers or cloud servers. Cloud platforms can be used to handle large-scale data storage, computing and analysis tasks. Cloud platforms can be, for example, quality inspection cloud platforms.

[0029] The following example uses a camera as the end device, an algorithm engine platform as the edge device, and a quality inspection cloud platform as the cloud platform. Figure 1 , a basic introduction to the traditional architecture of vehicle interior quality inspection system.

[0030] The camera can be fixed by a robotic arm. The path of the robotic arm is planned according to the vehicle model. The program is set to take photos of the vehicle interior, and the photos are transmitted to the algorithm engine platform through the industrial wired network for reasoning.

[0031] After receiving the photos, the algorithm engine platform performs inference based on the deep learning algorithm of the deployed artificial intelligence (AI) model, and transmits the inference results to the large screen at the line for display. At the same time, the results are sent to the quality inspection cloud platform via the company's internal Ethernet.

[0032] The quality inspection cloud platform is deployed with an AI model training platform. The images used for training are copied from the algorithm engine platform to the quality inspection cloud platform for training. Training requires data from hundreds of vehicles, and the transmission of photos is as much as 240G or more. After the training is completed, the updated algorithm needs to be deployed on the algorithm engine platform.

[0033] The traditional architecture of the above-mentioned vehicle interior quality inspection system has some problems. For example, the algorithm engine platform is connected to the robot arm and the camera through a wired network. Therefore, it needs to be deployed close to the robot arm and the camera, which will occupy the production site and is not conducive to the flexible adjustment of the production line. At the same time, with the increase of detection tasks, the scalability of the algorithm engine platform relying on local computing resources is limited. Adding new hardware equipment is not only costly, but also difficult to achieve dynamic allocation and optimization of resources, and the computing power of the algorithm engine platform cannot be shared with other tasks. For another example, the software update of the algorithm engine platform requires personnel to go to the site for operation and debugging, and the update of new functional software is slow, which makes software update and software management complicated. For another example, only part of the data is transmitted to the quality inspection cloud platform, and data between different devices and systems is difficult to share, resulting in data island problems. This limits data analysis and optimization across devices and factories, and cannot fully utilize the value of data. For another example, the network resources are fixed, and the image transmission strategy cannot be intelligently adjusted based on the status of the network channel. The main reason for the above problems is that the algorithm engine platform transmits data to the camera fixed to the robot arm and the quality inspection cloud platform through an industrial wired network.

[0034] For the above problems, relevant technologies propose to solve them through wireless networks. Taking 5G wireless networks as an example, with the maturity and promotion of 5G technology, industrial 5G will gradually become an important infrastructure for Industry 4.0 and smart manufacturing. Industrial 5G is the application of 5G technology in the industrial field, aiming to promote the innovative development of industrial automation, smart manufacturing, industrial Internet of Things, etc. 5G wireless networks have the characteristics of high bandwidth, low latency, and large-scale connection. 5G wireless networks have many advantages over wired networks in vehicle interior inspection. For example, the high bandwidth of 5G wireless networks can support the real-time transmission of high-definition and even ultra-high-definition images, ensuring that the quality inspection system can quickly receive and process visual data from the production line. For another example, 5G wireless networks can achieve efficient data transmission and computing task allocation between terminal devices, edge devices, and cloud-side devices. For example, refer to Figure 2 , the quality inspection system can dynamically adjust the allocation of computing power according to the current computing needs and network quality. Under high load conditions, edge devices can take over more processing tasks to relieve the pressure on cloud devices. This makes the 5G wireless network more efficient when transmitting vehicle interior quality inspection data. For another example, the quality of the 5G wireless network can be monitored. Wireless network quality can include real-time network quality or predicted network quality. There can be multiple criteria for determining wireless network quality, such as bandwidth, latency, jitter, packet loss rate, etc. Through real-time monitoring and prediction of wireless network quality, the industrial 5G system can work with industrial applications to take preventive measures before the network environment changes, thereby improving the resilience and stability of the entire system. This makes the 5G wireless network more reliable when transmitting vehicle interior quality inspection data.

[0035] Although there are many advantages in using 5G wireless networks to transmit vehicle interior quality inspection data, the 5G transmission capacity will change due to factors such as 5G network coverage, interference, and network congestion. Since the amount of data generated during the vehicle interior quality inspection process is very large, when the wireless network quality is poor, the images collected by the end-side device may be lost or the image frames may be lost during the transmission process to the cloud platform, which will affect the quality inspection accuracy of the vehicle interior.

[0036] In response to the above problems, this application proposes a method for quality inspection of vehicle interiors. That is, the transmission strategy of the current image data is determined based on the quality of the wireless network. The transmission strategy of the image data is determined in a targeted manner according to the quality of the wireless network, which can reduce the loss of vehicle interior quality inspection accuracy caused by poor network quality.

[0037] For ease of understanding, the following Figure 3 , the embodiments of the present application are illustrated in more detail.

[0038] Figure 3 A schematic flow chart of a method for quality inspection of vehicle interior provided in an embodiment of the present application. The method for quality inspection of vehicle interior can be applied to edge devices in a vehicle interior quality inspection system. The vehicle interior quality inspection system also includes an end-side device and a cloud platform connected to the edge device via a wireless network. The end-side device is used to collect image data of the vehicle interior. The cloud platform is used to perform quality inspection on the vehicle interior.

[0039] See also Figure 3 In step S310, image data of the vehicle interior is received from the terminal device.

[0040] The end-side device is connected to the edge device through a wireless network. The end-side device obtains the image data of the vehicle interior by taking pictures of the interior of the vehicle. The image data of the vehicle interior includes the images of the decoration and functional components inside the vehicle. The edge device receives the image data collected by the end-side device through the wireless network.

[0041] In step S320, a current transmission strategy for the picture data is determined from a plurality of transmission strategies according to the wireless network quality.

[0042] Wireless network quality can be divided into good wireless network quality and poor wireless network quality. Good wireless network quality means that the wireless network has no congestion, delay, jitter, packet loss, etc. There are many strategies for transmitting image data. For example, the edge device can transmit all received image data to the cloud platform, or the image transmission optimization module in the edge device can optimize the received image data first and transmit the optimized image data to the cloud platform. Optimizing image data can also include reducing the number of image transmissions, reducing the resolution or compression quality of the image, etc. Different transmission strategies correspond to different data transmission volumes.

[0043] When the wireless network quality is good, a transmission strategy can be used. For ease of description, the situation where the wireless network quality is good is referred to as the first condition, and the transmission strategy used when the wireless network quality meets the first condition is referred to as the first transmission strategy. When the wireless network quality is good, the adopted transmission strategy can be to transmit all collected pictures according to the original number of pictures and the original resolution of the pictures.

[0044] In the case of poor quality or congestion of the wireless network, another transmission strategy can be used. For ease of description, the situation of poor quality or congestion of the wireless network is referred to as the second condition below. The transmission strategy used when the quality of the wireless network meets the second condition is referred to as the second transmission strategy. The network quality corresponding to the first condition is higher than the network quality corresponding to the second condition. When the quality of the wireless network meets the second condition, the transmission strategy adopted may be, for example, only reducing the number of pictures to be transmitted, or only reducing the resolution of the pictures, or both reducing the number of pictures to be transmitted and reducing the resolution of the pictures. Reducing the number of pictures to be transmitted means that when the network quality is poor, the available broadband resources are limited, and the broadband resources can be used to preferentially transmit important pictures. Important pictures may be, for example, pictures containing defects or abnormalities. For pictures without defects or abnormalities, they may not be transmitted or delayed, which can reduce the number of pictures that need to be transmitted when the network quality is poor or congested, free up bandwidth for the transmission of important pictures, thereby reducing the transmission pressure, and thus reducing the loss of vehicle interior quality inspection accuracy caused by poor network quality. There are many strategies for reducing the resolution of pictures, for example, the pixels of all pictures can be compressed at a ratio, or the ratio of picture pixel compression can be selected according to the size of the component. When capturing images of a vehicle interior, there are usually multiple cameras. Figure 4As shown, there can be 6 cameras. Different cameras can capture images of components at different locations in the car. For example, cameras 1 and 2 can capture images of multiple components such as the left front door, the left rear door, the right front door, and the right rear door. Cameras 3 and 5 can capture images of multiple larger components such as the front windshield defrost cover and the carpet. Cameras 4 and 6 can capture images of multiple smaller components such as the auxiliary instrument armrest and the safety handle. The pixels of the compressed image can be compressed by a ratio of all the pixels of the image. For example, as shown in Table 1, the pixels of all the images can be processed by compressing 80%.

[0045] Table 1

[0046]

[0047] The pixels of the compressed image can also be compressed according to the size of the component. For relatively large components, the compression ratio of the image pixels can be higher. For relatively small components, the compression ratio of the image pixels can be lower. Continuing with Table 1 as an example, for the images of larger components captured by cameras 3 and 5, the pixels of the image can be compressed by 60%, or the pixels of the image can be compressed to 40% of the original. For the images of small components captured by cameras 4 and 6, the pixels of the image can be compressed by 20%, or the pixels of the image can be compressed to 80% of the original. After compressing the image pixels, the accuracy of vehicle interior quality inspection can reach more than 98.5%. It can be seen that by reducing the image resolution, the pressure on edge devices to transmit image data to the cloud platform can be reduced, thereby reducing the loss of vehicle interior quality inspection accuracy caused by poor network quality.

[0048] In step S330, at least part of the image data is transmitted to the cloud platform according to the current transmission strategy, so that the cloud platform performs quality inspection on the vehicle interior based on at least part of the data. At least part of the image data is transmitted to the cloud platform according to the current transmission strategy, and at least part of the data may include all the image data or part of the image data. When all the data in the image data is transmitted to the cloud platform for quality inspection, the edge device does not perform quality inspection on the image data. Since the edge device does not perform quality inspection on the image during the entire quality inspection process, the computing power deployment of the edge device can be reduced, and the size, cost and power consumption of the edge device can be reduced. Although the computing power deployed by the edge device is small, some simple calculations can still be performed. Therefore, in order to reduce the amount of data transmitted to the cloud platform, the image data to be transmitted to the cloud platform can also be preprocessed. Preprocessing can, for example, be a preliminary screening of defects or abnormalities in the image, preferentially transmitting defective or abnormal images, and reducing the number of images that need to be transmitted. Preprocessing can also be to reduce the resolution or compression quality of the image. The preprocessed data is then sent to the cloud platform for quality inspection, which can avoid the impact of network quality on the large number of transmitted images or high image resolution, thereby reducing the loss of quality inspection accuracy.

[0049] The above describes the case where all image data is transmitted to the cloud platform for quality inspection, and the edge device does not perform quality inspection on the image data. The following describes the case where the edge device performs quality inspection on the image data. When the edge device transmits only part of the image data to the cloud platform for quality inspection, the other part of the image data can be quality inspected by the edge device. For ease of description, the image data detected by the cloud platform may be referred to as the first part of data, and the image data detected by the edge device may be referred to as the second part of data. The edge device can perform quality inspection on the vehicle interior according to the second part of data, which can reduce the transmission requirements of the image data to the cloud platform and reduce the pressure on network transmission. In addition to performing quality inspection on the second part of data, the edge device can also optimize the image transmission of the first part of data to be transmitted to the cloud platform. Image transmission optimization can further reduce the amount of data transmitted to the cloud platform, reduce the pressure on network transmission, and thus reduce the loss of vehicle interior quality inspection accuracy caused by poor network quality.

[0050] When the edge device also performs quality inspection on the image, the amount of data transmitted to the cloud platform for quality inspection can be determined based on the quality of the wireless network. The amount of data processed by the cloud platform can be the amount of data after the edge device optimizes the image. When the network quality is good, the cloud platform processes more image data. For the sake of ease of description, the relatively good wireless network quality is referred to as the third condition below, and the amount of data processed by the cloud platform is referred to as the first data amount. When the network quality is relatively poor, the cloud platform processes less image data. For the sake of ease of description, the poor wireless network quality is referred to as the fourth condition below, and the amount of data processed by the cloud platform is referred to as the second data amount. The first data amount is greater than the second data amount. For example, when the network quality is good, the edge device can transmit 80% of the original image data collected by the camera to the cloud platform for quality inspection. When the network quality is poor, the edge device can transmit 60% of the original image data collected by the camera to the cloud platform for quality inspection. In order to reduce the amount of data transmitted to the cloud platform, the image transmission optimization module in the edge device can reduce the resolution of the image collected by the end-side device. Reducing the resolution can be achieved by compressing the image pixels. Again with Figure 4 Taking Table 1 as an example, to compress the image data volume transmitted to the cloud platform to 60% of the original detection image data volume, the images collected by cameras 1 and 2 can be compressed to 80% of the original pixels through edge devices, and the images collected by cameras 3 and 5 can be compressed to 40% of the original pixels through edge devices. The pixels of the images collected by cameras 4 and 6 are not compressed, but the edge devices perform quality inspection on the images collected by cameras 4 and 6. The edge devices perform quality inspection on the images collected by cameras 4 and 6, and compress the pixels of the images to be transmitted to the cloud platform for quality inspection, so that the accuracy of vehicle interior quality inspection can reach more than 98.5%. Therefore, by dynamically allocating resources and reducing the amount of data transmitted to the cloud platform, the loss of vehicle interior quality inspection accuracy caused by poor network quality can be reduced.

[0051] Optionally, in some embodiments, model training and model updates can be placed on the cloud platform. Compared with copying images from the edge device to the cloud platform for training each time, and then deploying them to the edge device after training, placing model training and model updates on the cloud platform can save labor costs.

[0052] The above article describes in detail how to determine the image data transmission strategy based on the wireless network quality. Wireless network quality can refer to the real-time wireless network quality or the predicted wireless network quality. Figure 5 This article describes in detail how to predict wireless network quality.

[0053] Figure 5 A schematic flowchart of a method for predicting wireless network quality provided in an embodiment of the present application.

[0054] Reference Figure 5In step S510, a wireless network quality prediction service is subscribed to a service module.

[0055] The service module may be a wireless network quality prediction module. The edge device may include an image transmission optimization module. The image transmission optimization module may send a request to the wireless network quality prediction module to subscribe to the wireless network quality prediction service. The wireless network quality prediction service may include network quality prediction data and prediction intervals. When the image transmission optimization module sends a request to the wireless network quality prediction module, the wireless network quality prediction module sends a request permission message and attaches parameters such as the prediction interval and prediction data. The wireless network quality prediction module may receive air interface information about the gateway from the base station, such as signal strength, signal-to-noise ratio, etc.

[0056] In step S520, the prediction information of the wireless network quality pushed by the service module is received.

[0057] The wireless network quality prediction module can send wireless network quality prediction information regularly according to the agreement. The transmission of wireless network quality prediction information can be implemented in various forms, for example, it can be Restful API, OPC / UA, MQTT, etc. After receiving the information sent by the wireless network quality prediction module, the image transmission optimization module in the edge device selects the corresponding image data transmission strategy.

[0058] For ease of understanding, the following Figure 6 The process of network quality prediction is introduced with examples in more detail.

[0059] Taking 5G wireless network as an example, the prediction of network quality can include the following process:

[0060] (1) Establish a 5G data connection between the industrial 5G gateway and the 5G core network to obtain an IP address;

[0061] (2) The 5G base station automatically and regularly sends the signal strength, signal-to-noise ratio, and scheduling information of the industrial 5G gateway's air interface to the 5G wireless network quality prediction module;

[0062] (3) The detection image transmission optimization module deployed on the algorithm engine platform sends a message request service to the 5G wireless network quality prediction module (IP address is IP2) according to the system configuration;

[0063] (4) The 5G wireless network quality prediction module sends a request permission message and attaches parameters such as prediction interval and prediction data;

[0064] (5) Check the image transmission optimization module to confirm that the service has started;

[0065] (6) The 5G wireless network quality prediction module sends wireless network quality prediction information periodically according to the agreement.

[0066] Combine the following Figure 7 and Figure 8 The embodiments of the present application are described in more detail by way of example. In the following example, the end-side device is a high-definition industrial camera fixed by a manipulator, the edge device is an industrial 5G algorithm engine platform, the 5G algorithm engine platform includes a detection image transmission optimization module, the cloud platform is a quality inspection cloud platform, the model training and updating are all performed on the quality inspection cloud platform, and the wireless network is an industrial 5G network.

[0067] Example (1): The algorithm engine platform does not perform quality inspection on image data. (Refer to Figure 7 )

[0068] Camera data collection: The robotic arm fixes the camera, plans the robotic arm path according to the vehicle model, sets the program to take pictures of the vehicle interior, and transmits the original inspection pictures taken to the industrial 5G algorithm engine platform.

[0069] The industrial 5G algorithm engine platform optimizes the detection images and transmits the optimized images to the quality inspection cloud platform through the 5G network: The industrial 5G algorithm engine platform includes a detection image transmission optimization module, which optimizes the data volume of the detection images according to the network quality information of the industrial 5G network. For example, reduce the number of images that need to be transmitted, and give priority to transmitting the most important detection images (for example, images that may contain defects or abnormalities). Or reduce the resolution or compression quality of the image to reduce the size of the image data. The industrial 5G network contains multiple gateways / terminals, as well as industrial 5G base stations and other equipment, supporting end-to-end wireless data communication functions for multiple services. The industrial 5G network predicts the 5G channel status and capacity for specific gateways / terminals, and provides the prediction results to the detection image transmission optimization module of the industrial 5G algorithm engine platform through the 5G gateway / terminal. The industrial 5G network transmits the transmission-optimized detection images to the quality inspection cloud platform.

[0070] The quality inspection cloud platform displays the quality inspection results and trains and updates the model: After receiving the photos, the quality inspection cloud platform performs inference based on the deployed deep learning algorithm, and uses the internal network of the enterprise to share the inference results to the required functional units, such as the line side large screen for display, quality inspection statistics unit, etc. The training platform is deployed on the quality inspection cloud platform (cloud), and is trained based on the transmitted quality inspection image data, and the detection model is optimized and updated in a timely manner.

[0071] Example (2): The algorithm engine platform performs quality inspection on image data. (Refer to Figure 8 )

[0072] Camera data collection: The robotic arm fixes the camera, plans the robotic arm path according to the vehicle model, sets the program to take pictures of the vehicle interior, and transmits the original inspection pictures taken to the industrial 5G algorithm engine platform.

[0073] The industrial 5G algorithm engine platform optimizes the detection images and transmits the optimized images to the quality inspection cloud platform through the 5G network: The industrial 5G algorithm engine platform includes a detection image transmission optimization module, which optimizes the data volume of the detection images according to the network quality information of the industrial 5G network. For example, the number of images that need to be transmitted is reduced, and the most important detection images (for example, images that may contain defects or abnormalities) are transmitted first. Or the resolution or compression quality of the image is reduced to reduce the image data size. At the same time, the 5G algorithm engine platform detection tasks can be increased to further reduce the image transmission requirements. The industrial 5G network includes multiple gateways / terminals, as well as industrial 5G base stations and other equipment, supporting end-to-end wireless data communication functions for multiple services. The industrial 5G network predicts the 5G channel status and capacity for specific gateways / terminals, and provides the prediction results to the detection image transmission optimization module of the industrial 5G algorithm engine platform through the 5G gateway / terminal. The industrial 5G network transmits the transmission optimized detection images to the quality inspection cloud platform. At the same time, the 5G algorithm engine platform sends the quality inspection results to the quality inspection cloud platform through the industrial 5G network.

[0074] The quality inspection cloud platform displays the quality inspection results and trains and updates the model: After receiving the photos and the quality inspection results sent by the 5G algorithm engine platform, the quality inspection cloud platform performs inference based on the deployed deep learning algorithm, and uses the internal network of the enterprise to share the inference results to the required functional units, such as the line side large screen for display, quality inspection statistics unit, etc. The training platform is deployed on the quality inspection cloud platform (cloud), and is trained according to the transmitted quality inspection image data, and the detection model is optimized and updated in a timely manner.

[0075] Combination of the above Figures 1 to 8 , described the method embodiment of the present application in detail, and the device embodiment of the present application is described in detail below. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the part not described in detail can refer to the previous method embodiment.

[0076] Fig. 9It is a structural schematic diagram of a device for quality inspection of vehicle interior provided by an embodiment of the present application. The device is deployed on an edge device in a vehicle interior quality inspection system. The vehicle interior quality inspection system also includes a terminal device and a cloud platform connected to the edge device through a wireless network. The terminal device is used to collect image data of the vehicle interior, and the cloud platform is used to perform quality inspection on the vehicle interior. The device includes a first communication module 910, a determination module 920, and a second communication module 930. The first communication module 910 is used to receive image data of the vehicle interior from the terminal device; the determination module 920 is used to determine the current transmission strategy of the image data from a plurality of transmission strategies according to the quality of the wireless network, and the plurality of transmission strategies correspond to a plurality of data transmission volumes respectively; the second communication module 930 is used to transmit at least part of the image data to the cloud platform according to the current transmission strategy, so that the cloud platform can perform quality inspection on the vehicle interior based on at least part of the data.

[0077] The embodiment of the present application also provides an edge device. The edge device includes a memory and a processor. The memory is used to store programs. The processor is used to execute the programs stored in the memory, so that the edge device executes the method for quality inspection of vehicle interior in any of the above embodiments.

[0078] An embodiment of the present application also provides a computer-readable storage medium, on which is stored a program for executing the method for quality inspection of vehicle interior in any of the above embodiments.

[0079] An embodiment of the present application also provides a vehicle interior quality inspection system, including: a terminal device, an edge device and a cloud platform, the edge device is connected to the terminal device and the cloud platform via a wireless network, and the edge device is used to execute a method for quality inspection of vehicle interior as described in any of the above embodiments.

[0080] It should be understood that in the embodiment of the present application, determining B based on A does not mean determining B only based on A, but B can also be determined based on A and / or other information.

[0081] It should be understood that in the embodiment of the present application, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0082] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0083] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0084] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0086] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0087] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can read or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. Available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, digital video discs (DVD)), or semiconductor media (eg, solid state disks (SSD)).

[0088] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for quality inspection of vehicle interior, characterized in that: The method is applied to an edge device in a vehicle interior quality inspection system, the vehicle interior quality inspection system further comprising a terminal device and a cloud platform connected to the edge device via a wireless network, the terminal device being used to collect image data of the vehicle interior, and the cloud platform being used to perform quality inspection on the vehicle interior, the method comprising: Receiving the image data of the vehicle interior from the terminal device; Determining, according to the quality of the wireless network, a current transmission strategy for the image data from a plurality of transmission strategies, wherein the plurality of transmission strategies correspond to a plurality of data transmission amounts respectively; At least part of the image data is transmitted to the cloud platform according to the current transmission strategy, so that the cloud platform can perform quality inspection on the vehicle interior based on the at least part of the data.

2. The method according to claim 1, characterized in that The determining, according to the wireless network quality, the current transmission strategy of the picture data from a plurality of transmission strategies includes: In response to the wireless network quality satisfying a first condition, using a first transmission strategy among the multiple transmission strategies; In response to the wireless network quality satisfying a second condition, using a second transmission strategy among the multiple transmission strategies; The wireless network quality corresponding to the first condition is higher than the wireless network quality corresponding to the second condition; The number of pictures transmitted by the first transmission strategy is greater than the number of pictures transmitted by the second transmission strategy; and / or, The resolution of the picture data transmitted by the first transmission strategy is greater than the resolution of the picture data transmitted by the second transmission strategy.

3. The method according to claim 1, characterized in that The terminal side device transmits all the data in the image data to the cloud platform.

4. The method according to any one of claims 1 to 3, characterized in that The transmitting at least part of the image data to the cloud platform according to the current transmission strategy includes: Preprocessing at least part of the data to determine at least part of the preprocessed data; Transmitting at least a portion of the preprocessed data to the cloud platform.

5. The method according to any one of claims 1 to 3, characterized in that The at least part of the data is a first part of the image data, the image data also includes a second part of the data, and the method further includes: Perform a quality inspection on the vehicle interior at the edge device based on the second portion of data.

6. The method according to claim 5, characterized in that The method further comprises: In response to the wireless network quality satisfying a third condition, determining that the data volume of the first part of data is a first data volume; In response to the wireless network quality satisfying a fourth condition, determining that the data amount of the first part of data is a second data amount; The wireless network quality corresponding to the third condition is higher than the wireless network quality corresponding to the fourth condition, and the first data volume is smaller than the second data volume.

7. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Subscribe to the prediction service of the wireless network quality to the service module; Receive the prediction information of the wireless network quality pushed by the service module.

8. A device for quality inspection of vehicle interior, characterized in that: The device is deployed on an edge device in a vehicle interior quality inspection system, and the vehicle interior quality inspection system also includes a terminal device and a cloud platform connected to the edge device via a wireless network, the terminal device is used to collect image data of the vehicle interior, and the cloud platform is used to perform quality inspection on the vehicle interior. The device includes: A first communication module, configured to receive the image data of the vehicle interior from the terminal device; A determination module, configured to determine a current transmission strategy for the image data from a plurality of transmission strategies according to wireless network quality, wherein the plurality of transmission strategies correspond to a plurality of data transmission amounts respectively; The second communication module is used to transmit at least part of the image data to the cloud platform according to the current transmission strategy, so that the cloud platform can perform quality inspection on the vehicle interior based on the at least part of the data.

9. An edge device, characterized in that: include: Memory, used to store programs; A processor, configured to execute the program stored in the memory so that the edge device executes the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores thereon data for executing the method according to any one of claims 1 to 7.

11. A vehicle interior quality inspection system, characterized in that: include: A terminal device, an edge device and a cloud platform, wherein the edge device is connected to the terminal device and the cloud platform via a wireless network, and the edge device is used to execute the method as described in any one of claims 1 to 7.

Images