Error-correctable VR image recognition method and system

Abstract

The invention discloses an error-correctable VR image recognition method and system, and belongs to the technical field of image processing, and the method comprises the steps: obtaining an original image when a user carries out a lane changing operation in a three-dimensional view field of VR equipment, and the three-dimensional view field is formed based on a real traffic scene in a preset region at the current moment; target vehicles are recognized based on the original images, the driving direction distance between the target vehicles and the current state information of the target vehicles are called, and the target vehicles comprise a user driving vehicle, a front vehicle and a rear vehicle; according to the driving direction distance between the target vehicles and the current state information of the target vehicles, triggering VR equipment to display a lane changing error correction prompt in a three-dimensional view field; the image recognition step is added according to the characteristics of the VR image, the error correction step is added on the basis of VR image recognition, and error correction of the user lane changing operation in the three-dimensional view field is achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more particularly to a VR image recognition method and system capable of error correction. Background Art

[0002] Panoramic cameras can output virtual reality (VR) images. Compared with traditional flat images, VR images have a wider viewing angle and richer information. However, it is precisely because of these characteristics of VR images that it is difficult to recognize VR images, and thus it is impossible to obtain accurate recognition results.

[0003] The Chinese patent with publication number CN116415019A provides a virtual reality VR image recognition method and device, electronic device, and storage medium. By obtaining one or more target VR expansion images, each target VR expansion image is cropped into multiple square sub-images, and each square sub-image is processed using the first attention classification model to obtain a classification result for each square sub-image, and then identify whether the target VR expansion image is a suspicious image, thereby realizing automatic recognition of suspicious images in VR images.

[0004] However, this patent can only identify VR expansion images and cannot be used to correct errors in vehicle driving operations, so the existing technology has shortcomings. Summary of the invention

[0005] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a correctable VR image recognition method and system, which can recognize the user's lane change operation based on the VR original image and correct the user's operation based on the image recognition result.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The present invention provides a VR image recognition method capable of error correction, which is executed by a server and includes:

[0008] Acquire an original image when a user performs a lane change operation in a three-dimensional field of view of a VR device, where the three-dimensional field of view is formed based on a real traffic scene in a preset area at a current moment;

[0009] Identify a target vehicle based on the original image, and retrieve the driving direction distance between the target vehicles and the current state information of the target vehicles, wherein the target vehicles include a user-driven vehicle, a front vehicle, and a rear vehicle, wherein the front vehicle is a vehicle located in front of the user-driven vehicle and adjacent to the user-driven vehicle on the target change road, and the rear vehicle is a vehicle located behind the user-driven vehicle and adjacent to the user-driven vehicle on the target change road;

[0010] According to the driving direction distance between the target vehicles and the current status information of the target vehicles, the VR device is triggered to display the lane change correction prompt in the three-dimensional field of view.

[0011] As a further improvement of the present invention, the target vehicle is identified based on the original image, comprising:

[0012] Based on the original image and the superpixel algorithm, generating a segmented image;

[0013] According to the feature information of each superpixel in the segmented image, superpixels with similar feature information are fused to form a plurality of superpixel regions, each of which represents an individual;

[0014] The target vehicle is identified according to feature information of each superpixel region, wherein the feature information of the superpixel region includes position, shape and texture features.

[0015] As a further improvement of the present invention, triggering the VR device to display the lane change error correction prompt in the three-dimensional field of view according to the driving direction distance between the target vehicles and the current state information of the target vehicles includes:

[0016] Calculating the travel distance of the target vehicle after completing the lane change according to the current state information;

[0017] If the sum of the first distance and the third distance is less than or equal to the fifth distance, and / or the sum of the second distance and the fifth distance is equal to the fourth distance, the VR device is triggered to display the lane change correction prompt in the three-dimensional field of view, the first distance is the distance in the driving direction between the vehicle in front and the vehicle driven by the user, the second distance is the distance in the driving direction between the vehicle behind and the vehicle driven by the user, the third distance is the driving distance of the vehicle in front, the fourth distance is the driving distance of the vehicle behind, and the fifth distance is the driving distance of the vehicle driven by the user.

[0018] As a further improvement of the present invention, the step of generating a segmented image based on the original image and the superpixel algorithm includes:

[0019] generating a corrected image according to the original image;

[0020] Selecting a plurality of cluster centers in the corrected image;

[0021] Performing an iterative operation, the iterative operation comprising: updating each cluster center according to the gradient value of each pixel point in the neighborhood corresponding to each cluster center, and classifying each pixel point in the neighborhood corresponding to each cluster center according to the distance between each pixel point in the neighborhood and the cluster center, repeating the above steps until the number of repetitions reaches a preset number of iterations, and obtaining the category corresponding to each pixel point in the corrected image;

[0022] The pixel points belonging to the same category are combined to obtain multiple super pixels.

[0023] As a further improvement of the present invention, the step of selecting a plurality of cluster centers in the corrected image includes:

[0024] Selecting a plurality of seed points in the rectified image;

[0025] Calculating the structural density of the corrected image, the structural density of the region where each seed point is located, and the average structural density of all the regions according to the gray value of each pixel in the corrected image;

[0026] Determining the complexity level of the region where each seed point is located according to the structural density of each corrected image, the structural density of the region where each seed point is located, and the average structural density of all the regions;

[0027] According to the complexity level of the region where each seed point is located, adjusting the number and position of the seed points in each region;

[0028] The cluster center is determined according to the adjusted number and position of each of the seed points.

[0029] As a further improvement of the present invention, the step of generating a corrected image according to the original image includes:

[0030] Sampling pixels of the original image to obtain a plurality of sampling points;

[0031] Obtaining the coordinates of each sampling point on the spherical domain;

[0032] Calculate the azimuth and polar angle of each sampling point according to the coordinates of each sampling point on the spherical domain;

[0033] Projecting each of the sampling points onto a plane according to the azimuth angle and polar angle of each of the sampling points to form a plane image;

[0034] The corrected image is obtained according to the planar image.

[0035] The present invention provides a VR image recognition method capable of error correction, which is executed by a VR device and includes:

[0036] The original image when the user performs a lane change operation in the three-dimensional field of view of the VR device is sent to the server, so that the server identifies the target vehicle according to the original image, retrieves the driving direction distance between the target vehicles and the current state information of the target vehicles, and triggers the VR device according to the driving direction distance between the target vehicles and the current state information of the target vehicles, and the three-dimensional field of view is formed based on the real traffic scene in the preset area at the current moment;

[0037] In response to the triggering of the server, a lane change correction prompt is displayed in the three-dimensional field of view.

[0038] The present invention provides a server, comprising:

[0039] Acquisition module: acquiring the original image when the user performs a lane change operation in the three-dimensional field of view of the VR device, where the three-dimensional field of view is formed based on the real traffic scene in the preset area at the current moment;

[0040] Identification module: Identify the target vehicle based on the original image, and retrieve the driving direction distance between the target vehicles and the current state information of the target vehicles, wherein the target vehicles include the user-driven vehicle, the front vehicle and the rear vehicle, wherein the front vehicle is the vehicle in front of the user-driven vehicle and adjacent to the user-driven vehicle on the target change road, and the rear vehicle is the vehicle behind the user-driven vehicle and adjacent to the user-driven vehicle on the target change road;

[0041] Trigger module: triggers the VR device to display the lane change error correction prompt in the three-dimensional field of view according to the driving direction distance between the target vehicles and the current state information of the target vehicles.

[0042] The present invention provides a VR device, comprising:

[0043] A sending module: sending an original image when a user performs a lane change operation in the three-dimensional field of view of the VR device to a server, so that the server identifies a target vehicle according to the original image, retrieves the driving direction distance between the target vehicles and the current state information of the target vehicles, and triggers the VR device according to the driving direction distance between the target vehicles and the current state information of the target vehicles, wherein the three-dimensional field of view is formed based on a real traffic scene in a preset area at the current moment;

[0044] A response module: responds to the trigger of the server and displays a lane change correction prompt in the three-dimensional field of view.

[0045] The present invention provides a VR image recognition system capable of error correction, comprising: a VR device and a server, wherein the VR device sends an original image when a user performs a lane change operation in a three-dimensional field of view of the VR device to the server, wherein the three-dimensional field of view is formed based on a real traffic scene in a preset area at the current moment;

[0046] The server identifies the target vehicle according to the original image, retrieves the driving direction distance between the target vehicles and the current state information of the target vehicles, and triggers the VR device according to the driving direction distance between the target vehicles and the current state information of the target vehicles;

[0047] The VR device responds to the triggering of the server and displays a lane change correction prompt in the three-dimensional field of view.

[0048] The present invention identifies the target vehicle based on the VR original image, and triggers the VR device according to the driving direction distance between the target vehicles and the current state information of the target vehicles, so that the VR device displays the lane change correction prompt to the user in time, which makes up for the shortcomings of the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of the method steps of the present invention;

[0050] Figure 2 A schematic diagram of a user performing a lane change operation in the present invention;

[0051] Figure 3 is a schematic diagram of the projection step in the present invention;

[0052] Figure 4 It is a schematic diagram of the system module of the present invention.

[0053] Figure numerals: 1. User driving vehicle; 2. Vehicle in front; 3. Vehicle behind. DETAILED DESCRIPTION

[0054] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations of the technical solution of the present invention.

[0055] The term "and / or" in the following text is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0056] And the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the drawings, and the words "bottom" and "top", "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively. The same parts are denoted by the same reference numerals.

[0057] In order to realize the recognition and error correction of the user's lane change operation in a virtual scene, the present invention provides a VR image recognition method and system with error correction.

[0058] like Figure 1 As shown, the embodiment of the present application provides a VR image recognition method with error correction, which is executed by a server and includes:

[0059] Obtaining the original image of the user performing a lane change operation in the three-dimensional field of view of the VR device, where the three-dimensional field of view is formed based on the real traffic scene in the preset area at the current moment;

[0060] Based on the original image, the target vehicle is identified, and the driving direction distance between the target vehicles and the current state information of the target vehicles are retrieved. The target vehicles include the user-driven vehicle 1, the front vehicle 2 and the rear vehicle 3. The front vehicle 2 is a vehicle located in front of the user-driven vehicle 1 and adjacent to the user-driven vehicle 1 on the target change road. The rear vehicle 3 is a vehicle located behind the user-driven vehicle 1 and adjacent to the user-driven vehicle 1 on the target change road.

[0061] According to the driving direction distance between the target vehicles and the current status information of the target vehicles, the VR device is triggered to display lane change correction prompts in the three-dimensional field of view.

[0062] The lane change operation refers to the behavior of a motor vehicle driving from the current lane into the adjacent lane, and the preset area can be an area selected by the user.

[0063] The correctable VR image recognition method provided in this embodiment can automatically recognize the acquired image after acquiring the image of the user performing a lane change operation, and trigger the VR device according to the result of the image recognition, so that the VR device will promptly display the lane change correction prompt to the user, so as to correct the user's lane change operation.

[0064] Furthermore, this embodiment provides an image recognition method, comprising:

[0065] Generate a segmented image based on the original image and superpixel algorithm;

[0066] According to the feature information of each superpixel in the segmented image, superpixels with similar feature information are fused to form multiple superpixel regions, each of which represents an individual.

[0067] The target vehicle is identified based on the feature information of each superpixel region, and the feature information of the superpixel region includes position, shape and texture features.

[0068] This embodiment fuses the segmented superpixels, and the multiple pixel areas after fusion are called superpixel areas, and multiple superpixel areas are obtained. This step can fuse the over-segmented superpixels into complete superpixel areas, solving the over-segmentation problem of the superpixel algorithm. At the same time, the fusion step also reduces the appearance of "pointed" superpixels. The characteristic of "pointed" superpixels is that the protruding part of the superpixel is pointed or the middle part is particularly thin, and the number of pixels contained in these parts is very small. If the fusion is not performed and the recognition is performed directly based on the superpixel, the accuracy of image recognition will be affected. Therefore, this embodiment performs superpixel fusion so that each superpixel area can fully represent an individual, and effectively identify the required target vehicle based on the position, shape and texture characteristics of each individual in the image.

[0069] Specifically, the feature information of each superpixel may include color features, texture features, etc., wherein the color features may be calculated using color information in RGB, HSV and / or CIELAB space, and the texture features represent the uniformity, detail and roughness of the image, which may be determined based on the grayscale of the image.

[0070] Furthermore, based on the image recognition results, the driving direction distance between the target vehicles and the current status information of the target vehicles can be retrieved, and based on the driving direction distance between the target vehicles and the current status information of the target vehicles, it can be judged whether the user's lane change operation is correct. If not, the VR device is triggered to display a lane change correction prompt in the three-dimensional field of view.

[0071] Specifically, the current state information of the target vehicle includes the speed of the vehicle 1 driven by the user. , the speed of the vehicle 2 in front , the speed of the rear vehicle 3 and the lane change angle of the user's vehicle , where the lane change angle Specifically, it is the angle between the vehicle body and the lane when the user performs a lane change operation.

[0072] like Figure 2 As shown, the specific judgment steps are:

[0073] According to the current state information of the target vehicle, the travel distance of the target vehicle after completing the lane change is calculated;

[0074] If the first distance The third distance The sum is less than or equal to the fifth distance , and / or the second distance Distance to the fifth The sum is equal to the fourth distance , triggering the VR device to display lane change correction prompts in the three-dimensional field of view, the first distance is the distance in the driving direction between the front vehicle 2 and the vehicle 1 driven by the user, and the second distance is the distance in the driving direction between the rear vehicle 3 and the vehicle 1 driven by the user, and the third distance is the driving distance of the front vehicle 2, the fourth distance is the driving distance of the rear vehicle 3, and the fifth distance is the driving distance of the vehicle 1 driven by the user.

[0075] The first distance The third distance The sum is less than or equal to the fifth distance Indicates that the user's vehicle 1 collides with the vehicle 2 in front; the second distance Distance to the fifth The sum is equal to the fourth distance Indicates that the user's vehicle 1 collides with the rear vehicle 3; if the first distance The third distance The sum is greater than the fifth distance , indicating that after the lane change operation, the user's vehicle 1 is still behind the front vehicle 2; if the second distance Distance to the fifth The sum is less than the fourth distance , indicating that during the lane change operation, the rear vehicle 3 drives in front of the user's vehicle 1, and after the lane change operation is completed, the user's vehicle 1 is behind the rear vehicle 3; if the second distance Distance to the fifth The sum is greater than the fourth distance , indicating that after the lane change operation, the user's driving vehicle 1 is still ahead of the front vehicle 2.

[0076] The specific calculation formula is:

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] This embodiment assumes that the target vehicle is traveling at a uniform speed. represents the distance between adjacent lanes, Indicates the travel time of the target vehicle.

[0082] Furthermore, this embodiment provides an image segmentation method, comprising:

[0083] Generate a rectified image based on the original image;

[0084] Select within the rectified image Cluster centers;

[0085] Performing an iterative operation, the iterative operation comprising: updating each cluster center according to the gradient value of each pixel point in the neighborhood corresponding to each cluster center, and classifying each pixel point in the neighborhood corresponding to each cluster center according to the distance between each pixel point in the neighborhood and the cluster center, repeating the above steps until the number of repetitions reaches a preset number of iterations, and obtaining the category corresponding to each pixel point in the corrected image;

[0086] Combine pixels belonging to the same category to get Superpixels.

[0087] This embodiment uses a superpixel algorithm to segment the corrected image, and uses the feature information of superpixels to replace internal pixels, which can effectively reduce redundancy in calculations. The cluster center is moved to the place with the smallest gradient in the neighborhood to avoid the cluster center falling on the contour boundary with a larger gradient, so as not to affect the subsequent clustering effect. The size of the area can be determined as four neighborhoods or eight neighborhoods according to actual needs.

[0088] Specifically, the calculation formula for the pixel gradient value is:

[0089] ;

[0090] in Pixel The gradient value at and Respectively Direction and Directional gradient.

[0091] Each pixel is classified according to the distance between the pixel and the cluster center in the neighborhood. Specifically, if a pixel is only in the neighborhood of one cluster center, the pixel is automatically classified into this category. If a pixel is in the neighborhood of multiple cluster centers, the distance between the pixel and multiple cluster centers needs to be calculated, and the cluster center with the smallest distance is selected and classified into its category.

[0092] The traditional K-means algorithm does not limit the neighborhood of the cluster center. Each cluster center will calculate the distance with all pixels in the image. The superpixel method provided in this embodiment limits the size of the neighborhood range on the basis of the K-means algorithm, effectively reducing the number of distance calculations and improving calculation efficiency.

[0093] For example, assuming that the pixel In In the neighborhood of cluster centers , pixel The distance from the cluster center is determined by the color difference component, the spatial position component and the weight parameter. The color difference component is calculated by the color information of the CIELAB color space. The specific calculation formula is:

[0094] ;

[0095] ;

[0096] ;

[0097] in, , Indicates Pixels and The distance between cluster centers, is the color difference component, is the component of spatial position, is the weight parameter, pixel The color components are , the pixel The coordinates after projection to the VR image are , similarly, The color component of the cluster center is , will The coordinates of the cluster centers after being projected onto the VR image are .

[0098] This embodiment uses CIELAB color to describe color components. Compared with traditional RGB color, CIELAB color can more accurately describe the color seen by the human eye in a digital way.

[0099] In the prior art, cluster centers are mostly generated uniformly, but such methods ignore the fact that content information in an image is unevenly distributed, resulting in that areas with complex contours in an image cannot be segmented in detail. To avoid the above problem, this embodiment provides a method for generating cluster centers, including:

[0100] Select multiple seed points in the rectified image;

[0101] According to the gray value of each pixel in the corrected image, the structural density of the corrected image, the structural density of the area where each seed point is located, and the average structural density of all areas are calculated;

[0102] Determine the complexity level of the region where each seed point is located according to the structural density of each corrected image, the structural density of the region where each seed point is located, and the average structural density of all regions;

[0103] According to the complexity level of the area where each seed point is located, adjust the number and location of seed points in each area;

[0104] The cluster center is determined based on the adjusted number and position of each seed point.

[0105] Common sampling methods include Hammersley sampling, Halton sampling, Fibonacci sampling, etc. This embodiment uses Fibonacci sampling, which can effectively alleviate the problem that the distance between the original seed points selected in Hammersley sampling and Halton sampling is too close or too far.

[0106] The specific calculation formula is:

[0107] ;

[0108] ;

[0109] ;

[0110] in, is the number of pixels in the corrected image, is the number of selected seed points, For the The structural density of the region where the seed point is located, To correct the structural density of the image, is the average structural density of all regions, Indicates Seed points, Seed point Area The gray value is less than the fixed threshold The number of pixels, It is a region The total number of pixels in The gray value in the corrected image is less than a fixed threshold The number of pixels, preferably a fixed threshold .

[0111] Seed Point Area The scope included is:

[0112] ;

[0113] in, express The scope of inclusion, express The horizontal axis range is express The vertical axis range is Seed point The coordinates of is the size of the seed point, For the plane image The polar angle of the line, is a factor that adjusts the degree of overlap between adjacent regions. .

[0114] definition , , if the structural density of the current area , indicating that the content of this area is highly complex. , indicating that the content complexity of this area is low. , indicating that the content complexity of this area is average.

[0115] If the complexity of the regional content is high, the number of seed points in the region is increased; if the complexity of the regional content is low, the number of seed points in the region is deleted; if the complexity of the regional content is average, the number of seed points in the region remains unchanged. After adjustment, the final seed point is used as the cluster center.

[0116] This embodiment can determine the content complexity of the region based on the structural density of the region, and dynamically adjust the number of seed points in the local region according to the different content complexities, thereby obtaining the final required clustering center, thereby achieving the purpose of making the position and number of clustering centers adaptive to the image content information.

[0117] Furthermore, this embodiment provides a method for generating a corrected image, such as Figure 3 As shown, including:

[0118] The pixels of the original image are sampled to obtain multiple sampling points;

[0119] Get the coordinates of each sampling point on the spherical domain;

[0120] Calculate the azimuth and polar angle of each sampling point according to the coordinates of each sampling point on the spherical domain;

[0121] Projecting each sampling point onto a plane according to the azimuth and polar angle of each sampling point to form a plane image;

[0122] A rectified image is obtained based on the planar image.

[0123] Unlike traditional plane images, the data format of VR images during display and storage is not unified. The processing of VR images requires projecting the VR images onto a plane. Therefore, this embodiment maps the longitude of the VR image to equally spaced vertical lines and maps the latitude circle to equally spaced horizontal lines. The plane image obtained after projection is more in line with the way people view it and is also easier to process the image.

[0124] For example, in the spherical domain The coordinates of the points are ,Will The point after projecting the point onto the plane is denoted as , and the coordinates projected onto the plane are labeled , the azimuth is recorded as , the polar angle is recorded as , The distance from the point to the center of the sphere is , then according to:

[0125] ;

[0126] ;

[0127] You can get:

[0128] ;

[0129] ;

[0130] The coordinates on the spherical domain are the coordinates in the VR original image. is the width of the plane image, is the height of the plane image.

[0131] Correspondingly, the formula for projecting plane coordinates to the spherical domain is:

[0132] ;

[0133] However, the plane image obtained by the above projection method is prone to distortion, so this embodiment further provides a distortion correction method to correct the plane image, specifically:

[0134] ;

[0135] in Express The corrected coordinates, , , and represents rotation and scaling changes, and Represents translation change, parameter , , , , and It can be obtained by the least squares method.

[0136] This embodiment converts the original image into a corrected image through image projection and image correction steps, which serves as the basis for subsequent image processing.

[0137] From the above analysis, it can be seen that the correctable VR image recognition method provided in this embodiment converts the VR image into an easy-to-process plane image through projection, and corrects the image deformity caused by the projection, then uses superpixels to replace internal pixels for image recognition, and retrieves corresponding information based on the image recognition result. Finally, the VR device is triggered to display a lane change correction prompt based on the retrieved information, thereby forming a recognition and error correction method suitable for VR images.

[0138] Furthermore, the embodiment of the present application provides a VR image recognition method with error correction, which is executed by a VR device and includes:

[0139] The original image when the user performs a lane change operation in the three-dimensional field of view of the VR device is sent to the server, so that the server can identify the target vehicle according to the original image, retrieve the driving direction distance between the target vehicles and the current state information of the target vehicles, and trigger the VR device according to the driving direction distance between the target vehicles and the current state information of the target vehicles, and the three-dimensional field of view is formed based on the real traffic scene in the preset area at the current moment;

[0140] Respond to server triggers and display lane change correction prompts in the three-dimensional field of view.

[0141] Furthermore, an embodiment of the present application provides a server, including:

[0142] Acquisition module: acquires the original image when the user performs a lane change operation in the three-dimensional field of view of the VR device. The three-dimensional field of view is formed based on the real traffic scene in the preset area at the current moment;

[0143] Identification module: Identify the target vehicle based on the original image, and retrieve the driving direction distance between the target vehicles and the current state information of the target vehicles. The target vehicles include the user-driven vehicle 1, the front vehicle 2 and the rear vehicle 3. The front vehicle 2 is the vehicle in front of the user-driven vehicle 1 and adjacent to the user-driven vehicle 1 on the target change road. The rear vehicle 3 is the vehicle behind the user-driven vehicle 1 and adjacent to the user-driven vehicle 1 on the target change road.

[0144] Trigger module: Based on the driving direction distance between the target vehicles and the current status information of the target vehicles, the VR device is triggered to display lane change correction prompts in the three-dimensional field of view.

[0145] Furthermore, an embodiment of the present application provides a VR device, including:

[0146] Sending module: Sending the original image of the user performing the lane change operation in the three-dimensional field of view of the VR device to the server, so that the server can identify the target vehicle according to the original image, retrieve the driving direction distance between the target vehicles and the current state information of the target vehicles, and trigger the VR device according to the driving direction distance between the target vehicles and the current state information of the target vehicles. The three-dimensional field of view is formed based on the real traffic scene in the preset area at the current moment;

[0147] Response module: responds to the server's trigger and displays lane change correction prompts in the three-dimensional field of view.

[0148] Further, such as Figure 4 As shown, an embodiment of the present application provides a correctable VR image recognition system, including: a VR device and a server, wherein the VR device sends an original image when a user performs a lane change operation in a three-dimensional field of view of the VR device to the server, and the three-dimensional field of view is formed based on a real traffic scene in a preset area at the current moment;

[0149] The server identifies the target vehicle based on the original image, retrieves the driving direction distance between the target vehicles and the current state information of the target vehicles, and triggers the VR device based on the driving direction distance between the target vehicles and the current state information of the target vehicles;

[0150] The VR device responds to the server's trigger and displays lane change correction prompts in the three-dimensional field of view.

[0151] For example, through VR traffic safety simulation equipment, users can experience the process of road driving in a virtual road driving scene. However, when the user's driving operation is wrong and causes a traffic accident, because the user is in a virtual scene, he cannot truly feel the occurrence of the accident. At this time, it is necessary to remind the user in an appropriate way. For example, when the user changes lanes, the VR device will promptly collect and send the user's operation image to the server. The server will judge the accuracy of the user's operation based on the image recognition method, and promptly trigger the VR device to display a lane change correction prompt to enhance the user's experience and safe driving awareness.

[0152] An error-correctable VR image recognition method and system are provided in an embodiment of the present application. Based on the characteristics that VR images are prone to distortion and data processing is difficult, image correction and superpixel segmentation steps are added to the existing image recognition steps, providing a recognition method that is more suitable for VR images, and adding an error correction step based on the image recognition result to achieve error correction of the user's lane change operation.

[0153] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A VR image recognition method with error correction capability, characterized in that: Executed by the server, including: Acquire an original image when a user performs a lane change operation in a three-dimensional field of view of a VR device, where the three-dimensional field of view is formed based on a real traffic scene in a preset area at a current moment; Identify a target vehicle based on the original image, and retrieve the driving direction distance between the target vehicles and the current state information of the target vehicles, wherein the target vehicles include a user-driven vehicle, a front vehicle, and a rear vehicle, wherein the front vehicle is a vehicle located in front of the user-driven vehicle and adjacent to the user-driven vehicle on the target change road, and the rear vehicle is a vehicle located behind the user-driven vehicle and adjacent to the user-driven vehicle on the target change road; According to the driving direction distance between the target vehicles and the current state information of the target vehicles, triggering the VR device to display a lane change error correction prompt in the three-dimensional field of view; The method of triggering the VR device to display a lane change error correction prompt in the three-dimensional field of view according to the driving direction distance between the target vehicles and the current state information of the target vehicles includes: Calculating the travel distance of the target vehicle after completing the lane change according to the current state information; If the sum of the first distance and the third distance is less than or equal to the fifth distance, and / or the sum of the second distance and the fifth distance is equal to the fourth distance, the VR device is triggered to display the lane change correction prompt in the three-dimensional field of view, the first distance is the distance in the driving direction between the vehicle in front and the vehicle driven by the user, the second distance is the distance in the driving direction between the vehicle behind and the vehicle driven by the user, the third distance is the driving distance of the vehicle in front, the fourth distance is the driving distance of the vehicle behind, and the fifth distance is the driving distance of the vehicle driven by the user.

2. The error-correctable VR image recognition method according to claim 1, characterized in that: The identifying the target vehicle based on the original image comprises: Based on the original image and the superpixel algorithm, generating a segmented image; According to the feature information of each superpixel in the segmented image, superpixels with similar feature information are fused to form a plurality of superpixel regions, each of which represents an individual; The target vehicle is identified according to feature information of each superpixel region, wherein the feature information of the superpixel region includes position, shape and texture features.

3. The error-correctable VR image recognition method according to claim 2, characterized in that: The step of generating a segmented image based on the original image and the superpixel algorithm comprises: generating a corrected image according to the original image; Selecting a plurality of cluster centers in the corrected image; Performing an iterative operation, the iterative operation comprising: updating each cluster center according to the gradient value of each pixel point in the neighborhood corresponding to each cluster center, and determining the category of each pixel point in the neighborhood corresponding to each cluster center according to the distance between each pixel point in the neighborhood and the cluster center, repeating the above steps until the number of repetitions reaches a preset number of iterations, and obtaining the category corresponding to each pixel point in the corrected image; The pixel points belonging to the same category are combined to obtain multiple super pixels.

4. The error-correctable VR image recognition method according to claim 3, characterized in that: The selecting a plurality of cluster centers in the corrected image comprises: Selecting a plurality of seed points in the rectified image; Calculating the structural density of the corrected image, the structural density of the region where each seed point is located, and the average structural density of all the regions according to the gray value of each pixel in the corrected image; Determining the complexity level of the region where each seed point is located according to the structural density of each corrected image, the structural density of the region where each seed point is located, and the average structural density of all the regions; According to the complexity level of the region where each seed point is located, adjusting the number and position of the seed points in each region; The cluster center is determined according to the adjusted number and position of each of the seed points.

5. The error-correctable VR image recognition method according to claim 3, characterized in that: The step of generating a corrected image according to the original image comprises: Sampling pixels of the original image to obtain a plurality of sampling points; Obtaining the coordinates of each sampling point on the spherical domain; Calculate the azimuth and polar angle of each sampling point according to the coordinates of each sampling point on the spherical domain; Projecting each of the sampling points onto a plane according to the azimuth angle and polar angle of each of the sampling points to form a plane image; The corrected image is obtained according to the planar image.

6. A VR image recognition method with error correction capability, characterized in that: Executed by VR devices, including: The original image when the user performs a lane change operation in the three-dimensional field of view of the VR device is sent to the server, so that the server identifies the target vehicle according to the original image, retrieves the driving direction distance between the target vehicles and the current state information of the target vehicles, and triggers the VR device according to the driving direction distance between the target vehicles and the current state information of the target vehicles, and the three-dimensional field of view is formed based on the real traffic scene in the preset area at the current moment; In response to the triggering of the server, a lane change correction prompt is displayed in the three-dimensional field of view.

7. A server, applied to the error-correctable VR image recognition method according to claim 1, characterized in that: The server comprises: An acquisition module, which acquires an original image when a user performs a lane change operation in a three-dimensional field of view of a VR device, wherein the three-dimensional field of view is formed based on a real traffic scene in a preset area at a current moment; an identification module, identifying a target vehicle based on the original image, and retrieving a driving direction distance between the target vehicles and current state information of the target vehicles, wherein the target vehicles include a user-driven vehicle, a front vehicle, and a rear vehicle, wherein the front vehicle is a vehicle located in front of the user-driven vehicle and adjacent to the user-driven vehicle on the target change road, and the rear vehicle is a vehicle located behind the user-driven vehicle and adjacent to the user-driven vehicle on the target change road; The trigger module triggers the VR device to display the lane change error correction prompt in the three-dimensional field of view according to the driving direction distance between the target vehicles and the current state information of the target vehicles.

8. A VR device, applied to the error-correctable VR image recognition method according to claim 6, characterized in that: The VR device includes: a sending module, which sends the original image when the user performs a lane change operation in the three-dimensional field of view of the VR device to the server, so that the server identifies the target vehicle according to the original image, retrieves the driving direction distance between the target vehicles and the current state information of the target vehicles, and triggers the VR device according to the driving direction distance between the target vehicles and the current state information of the target vehicles, and the three-dimensional field of view is formed based on the real traffic scene in the preset area at the current moment; A response module responds to the triggering of the server and displays a lane change correction prompt in the three-dimensional field of view.

9. A VR image recognition system capable of error correction, characterized in that: include: A VR device and a server, wherein the VR device sends an original image of a user performing a lane change operation in a three-dimensional field of view of the VR device to the server, wherein the three-dimensional field of view is formed based on a real traffic scene in a preset area at the current moment; The server identifies the target vehicle according to the original image, retrieves the driving direction distance between the target vehicles and the current state information of the target vehicles, and triggers the VR device according to the driving direction distance between the target vehicles and the current state information of the target vehicles; The VR device responds to the triggering of the server and displays a lane change correction prompt in the three-dimensional field of view.

Images